-- Logs begin at Thu 2022-07-21 16:51:02 UTC, end at Thu 2022-07-21 18:26:27 UTC. -- Jul 21 17:19:38.723671 dv-cinder-825492-27-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 17:19:38.731662 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 17:19:38 2022] *** Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: nodename: dv-cinder-825492-27-iscsi Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: machine: x86_64 Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: clock source: unix Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: pcre jit disabled Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: detected number of CPU cores: 4 Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: current working directory: / Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: detected binary path: /usr/bin/uwsgi-core Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: your processes number limit is 47680 Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: your memory page size is 4096 bytes Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: detected max file descriptor number: 2048 Jul 21 17:19:38.735005 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: lock engine: pthread robust mutexes Jul 21 17:19:38.735705 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: thunder lock: enabled Jul 21 17:19:38.735705 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 17:19:38.735705 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 17:19:38.778571 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: Python main interpreter initialized at 0x559b9006c9c0 Jul 21 17:19:38.778571 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: python threads support enabled Jul 21 17:19:38.778571 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: your server socket listen backlog is limited to 100 connections Jul 21 17:19:38.778571 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: your mercy for graceful operations on workers is 90 seconds Jul 21 17:19:38.778948 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 17:19:38.779093 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: *** Operational MODE: preforking *** Jul 21 17:19:38.779372 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: *** uWSGI is running in multiple interpreter mode *** Jul 21 17:19:38.779372 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: spawned uWSGI master process (pid: 80206) Jul 21 17:19:38.779575 dv-cinder-825492-27-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 17:19:38.780250 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: spawned uWSGI worker 1 (pid: 80210, cores: 1) Jul 21 17:19:38.781827 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: spawned uWSGI worker 2 (pid: 80211, cores: 1) Jul 21 17:19:38.781827 dv-cinder-825492-27-iscsi devstack@keystone.service[80206]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 17:19:44.228428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 17:19:44.228428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: from cryptography.utils import int_from_bytes Jul 21 17:19:44.229178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 17:19:44.229178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: from cryptography.utils import int_from_bytes Jul 21 17:19:44.229869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 17:19:44.229869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: from cryptography.utils import int_from_bytes Jul 21 17:19:44.230314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 17:19:44.230314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: from cryptography.utils import int_from_bytes Jul 21 17:19:44.278091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 17:19:44.278291 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 17:19:44.278566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] command line args: [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 17:19:44.278746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 17:19:44.278852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 17:19:44.278941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 17:19:44.279040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ================================================================================ {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 17:19:44.279117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] command line args: [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 17:19:44.279302 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] admin_token = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.279384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 17:19:44.279550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] config_dir = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.279634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ================================================================================ {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 17:19:44.279738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.279823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] admin_token = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.279922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] config_source = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.280040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] config_dir = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.280115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] debug = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.280231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.280334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.280556 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] config_source = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.280619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] debug = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.280696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.280831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.280962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.281059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.281152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.281252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.281346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.281445 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.281543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.282435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.282576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_config_append = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.282576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.282769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.282907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_dir = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.282981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_config_append = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_file = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_dir = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283361 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_file = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.283991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.284970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.285088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.285168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.285242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.285323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.285439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.285578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.285696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.285837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.285957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.286233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.286426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] publish_errors = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.286550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.286669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] publish_errors = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.286778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.286895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.287003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.287114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.287229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.287340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.287450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.287580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.287699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.287810 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.287920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] standard_threads = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.288030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] standard_threads = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.288192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.288327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.288440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.288562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.288674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.288789 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.288912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] use_journal = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.289024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] use_journal = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.289024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] use_json = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.289229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] use_json = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.289338 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] use_stderr = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.289447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] use_stderr = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.289565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] use_syslog = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.289676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] use_syslog = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.289786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.289895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 17:19:44.290010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.290133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.290256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.290373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.290484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.290601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.290712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.290833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.290944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.291060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.291176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.291292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.291402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.291519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.291631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.291740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.291850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.291959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.292074 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.292192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.292318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.292431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.292540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.292651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.292760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.292760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.292983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.293098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.293216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.293337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.293454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.294130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.294249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.294249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.294463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.294574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.294685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.294861 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.294977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.295126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.295239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.295384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.295496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.295647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.295766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.295904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.296018 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.296156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.296271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.296448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.296562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.296678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.296798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.296910 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.297020 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.297143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.297255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.297365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.297474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.297583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.297736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.297852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.297990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.298104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.298241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.298419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.298542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.298677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.298798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.298952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.299095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.299206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.299324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.299436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.299545 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.299654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.299763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.299872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.300061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.300181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.300310 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.300427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.300547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.300657 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.300778 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.300891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.301007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.301125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.301236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.301346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.301448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.301549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.301659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.301824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.301939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.302767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.302896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.302994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.303104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.303221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.303323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.303438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.303566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.303679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.303795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.303906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.304018 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.304159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.304286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.304422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.304543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.304662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.304771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.304882 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.305000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.305103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.305213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.305330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.305447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.305559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.305686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.305801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.305912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.306030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.306143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.306259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.306372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.306490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.306608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.306711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.306821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.306943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.307055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.307172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.307292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.307395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.307505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.307615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.307731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.307842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.307964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.308077 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.308187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.308339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.308454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.308565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.308688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.308801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.308918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.309035 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.309187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.309838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.309956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.310057 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.310167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.310278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.310379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.310493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.310589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.310682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.310682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.310888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.310998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.311117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.311230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.311340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.311456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.311568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.311679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.311797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.311909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.312019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.312129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.312239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.313636 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.313836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.313954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.314065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.314178 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.314289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.314400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.314400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.314627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.314753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.314753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.314964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.315075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.315075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.315283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.315402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.315520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.315638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.315751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.315862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.315978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.315978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.316197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.316333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.316449 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.316572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.316686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.external = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.316806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.316919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.317029 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.317140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.317252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.317371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.317483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.317594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.password = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.317704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.317816 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.token = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.317933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.318052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.318196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.318318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.318439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.318558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.318672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.external = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.318783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.318896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.319008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.319132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.319247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.319358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.319358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.password = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.319576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.319697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.token = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.319811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.319924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.320042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.320155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.320266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.320408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.320529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.320648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.320761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.320879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.320993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.321112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.321226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.321345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.321345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.321573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.321687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.321807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.321921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.322034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.322146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.322257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.322368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.322488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.322601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.322713 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.322824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.322936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.323054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.323175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.323288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.323288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.323498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.323610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.323721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.323721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.323940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.323940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.324476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.324601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.324716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.324909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.325025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.325214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.325330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.325467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.325583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.325707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.325821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.325933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.326043 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.326193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.326311 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.326431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.326550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.326663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.326775 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.326887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.326998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.327118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.327232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.327354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.327466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.327578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.327689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.327878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.327994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.328106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.328218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.328352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.328467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.328593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.328707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.328819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.328930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.329042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.329161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.329275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.329388 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.329562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.329678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.329789 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.329909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.330023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.330135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.330269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.330387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.330521 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.330646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.330760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.330871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.330983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.331095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.331207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.331327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.331442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.331557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.331669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.331781 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.331893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.332012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.332153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.332270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.332437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.332551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.332663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.332785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.332899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.333012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.333124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.333235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.333347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.333466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.333581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.333694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.333812 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.333925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.334037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.334155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.334268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.334380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.334498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.334612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.334724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.334843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.334957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.335069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.335181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.335297 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.335410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.335530 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.335645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.335757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.335869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.335980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.336092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.336211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.336342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.336457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.336568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.336681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.336804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.336926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.337039 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.337150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.337261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.337371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.337488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.337609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.337722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.337839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.337952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.338062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.338173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.338291 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.338404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.338516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.338628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.338739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.338851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.338971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.339084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.339196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.339307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.339419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.339530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.339650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.339763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.339876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.339987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.340096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.340242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.340427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.340594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.340713 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.340713 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.340941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.340941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.341162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.341274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.341386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.341501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.341708 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.341836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.341951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.342063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.342174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.342285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.342396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.342516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.342630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.342740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.342853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.342964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.343075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.343194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.343307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.343418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.343529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.343641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.343751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.343870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.343982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.344095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.344206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.344336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.344452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.344575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.344689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.344808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.344921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.345033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.345151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.345273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.345386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.345497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.345609 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.345719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.345830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.345950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.346063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.346175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.346286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.346397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.346508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.346628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.346741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.346853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.346963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.347075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.347187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.347307 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.347420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.347537 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.347649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.347761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.347872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.347991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.348104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.348216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.348367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.348482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.348594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.348716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.348830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.348941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.349052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.349163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.349274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.349393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.349506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.349616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.349727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.349837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.349948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] role.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.350129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.350245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] role.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.350356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.350467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.350578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.350689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.350808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.350921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.351033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.351144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.351262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] role.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.351375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.351498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] role.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.351612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.351724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.351836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.351947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.352058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.352177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.352308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.352424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.352536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.352647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.352770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.352884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.353012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.353124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.353235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.353346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.353466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.353580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.353692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.353807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.353920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.354031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.354150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.354263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.354375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.354486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.354598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.354709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.354911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.355028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.355139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.355249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.355383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.355500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.355621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.355733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.355850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.356531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.356688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.356805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.356930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.357044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.357155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.357266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.357377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.357488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.357607 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.357722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.357833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.357944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.358055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.358165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.358285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.358388 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.358483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.358613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.358731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.358852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.358965 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.359076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.359187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.359298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.359409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.359527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.359641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.359784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.359901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.360018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.360131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.360250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.360383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.360518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.360634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.360768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.360885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.361009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.361124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.361235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.361346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.361457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.361567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.361686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.361799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.361912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.362023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.362160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.362276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.362395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.362508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.362620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.362730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.362841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.362953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.363096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.363214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.connection = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.363335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.363456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.363570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.363682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.363853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.363971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.364091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.364204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.connection = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.364360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.364486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.364600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.364712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.364823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.364943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.365058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.365169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.365281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.365392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.365504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.365628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.365742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.365853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.365966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.366078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.366190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.366309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.366422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.366533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.366645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.366756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.366868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.366988 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.367102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.367213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.367325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.367436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.367548 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 17:19:44.367667 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.367780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.367998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.368235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 17:19:44.368431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 17:19:44.398256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.398488 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.398578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.398713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.398799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.398942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.407110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.407477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.435816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.436038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.436528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.436769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.488966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.577151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.613798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.614034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.639434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.639785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.680576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.680802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.681027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.681249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.681467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.681681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.696007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.696242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.696495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.696720 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.696941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.697162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.706747 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.706982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.707204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.707423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.707639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.707858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.708064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.710295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.710530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.710756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.710977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.711195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.711417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.711634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.719658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.720741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.732908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.733140 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.733363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.733583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.733798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.734013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.734228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.735247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.735488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.735719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.735942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.736161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.736406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.736624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 17:19:44.751594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.751737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.751857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 17:19:44.751975 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 17:19:44.752101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.752218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.752365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 17:19:44.752495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 17:19:44.752612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.752727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.752850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.752967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.753082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.753267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.753413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.753679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.764483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.764730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.764852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.765008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.765127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.765278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.765399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.765558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.773438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.773715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.776754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.777032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.777151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.777314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.777433 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.777704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.777824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.778507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.778624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.778770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.779053 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.779053 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.779274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.779397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.779514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.779628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.779752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.779869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.779984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.780105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.780221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.780369 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.807301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.807566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.808944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.809325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.809519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.809758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.814616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.814896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.815177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.815325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.815611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.815898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.822516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.822798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.823380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.823664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.827725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.828008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.828289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.828582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.828716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.828855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.828974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.829125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.829244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.829390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.829521 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.829637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.829753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.829932 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.830202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.830470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.848961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.849236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.849719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.849998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.853936 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.854542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.857811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.858103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.858361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.858658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.863141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.863651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.866780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.867417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.870587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.870888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.871173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.871377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.871497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.872217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.872369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.872536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.872656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.872869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.873167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.873451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.892102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.892500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.892772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.893047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.893327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.893612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.893883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.894160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.901026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.902130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.902735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.903032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.904084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.904386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.906140 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.906418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.906692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.906980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.907263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.907457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.907741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.908025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.908326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.908740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.921362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.921641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.921922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.922212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.922492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.922613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.922784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.922908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.923068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.923191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.923424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.923704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.923993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.924270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.943118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.943279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.946417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.946546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.950278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.950431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.951953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.952075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.953625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.953763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.953989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.954112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.954278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.954530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.961732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.963491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.966793 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.966943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.967062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.967346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.967627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.969373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.974652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.975064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.975471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.975865 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.976916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.977204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.977482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.986025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.986365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.986580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.986857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.987130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.987408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.987682 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.987957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.988229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.988521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.988809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.989073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.989336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.989602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.990267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:44.990569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:44.990853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.000532 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 17:19:45.000818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.001099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.001377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.001695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.001941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.002217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.002494 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.002773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.003051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.004587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.004883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.005169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.005455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 17:19:45.028457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559b9006c9c0 pid: 80211 (default app) Jul 21 17:19:45.031852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea419adf-110a-4c79-a64c-19fa07e80a6a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:45.032157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea419adf-110a-4c79-a64c-19fa07e80a6a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:45.032469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea419adf-110a-4c79-a64c-19fa07e80a6a None None] PATH_INFO: `/v3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:45.036646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Thu Jul 21 17:19:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:19:45.045094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559b9006c9c0 pid: 80210 (default app) Jul 21 17:19:52.726432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edca17-5749-4dde-98bd-aee66a501013 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:52.727008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edca17-5749-4dde-98bd-aee66a501013 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:52.727008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edca17-5749-4dde-98bd-aee66a501013 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:52.728363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 2/2] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:19:52.736714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:52.737129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:52.737277 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:52.808125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: INFO dbcounter [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] Registered counter for database keystone Jul 21 17:19:52.829830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG oslo_db.sqlalchemy.engines [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80210) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 17:19:52.835945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writer thread running {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 17:19:53.012257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG passlib.handlers.bcrypt [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80210) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 17:19:53.012986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG passlib.handlers.bcrypt [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80210) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 17:19:53.067256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:19:53.068905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:19:53.191837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1/3] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:19:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 17:19:53.398493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edca17-5749-4dde-98bd-aee66a501013 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:53.468987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: INFO dbcounter [None req-e3edca17-5749-4dde-98bd-aee66a501013 None None] Registered counter for database keystone Jul 21 17:19:53.488632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG oslo_db.sqlalchemy.engines [None req-e3edca17-5749-4dde-98bd-aee66a501013 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80211) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 17:19:53.494721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writer thread running {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 17:19:53.613959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edca17-5749-4dde-98bd-aee66a501013 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:53.618242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3e1f3-0111-4e1b-bbb3-f394cacc41e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:53.618911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3e1f3-0111-4e1b-bbb3-f394cacc41e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:53.619196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3e1f3-0111-4e1b-bbb3-f394cacc41e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:53.619471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3e1f3-0111-4e1b-bbb3-f394cacc41e5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:53.627510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3e1f3-0111-4e1b-bbb3-f394cacc41e5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:53.679209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: WARNING keystone.common.rbac_enforcer.enforcer [None req-64a3e1f3-0111-4e1b-bbb3-f394cacc41e5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 17:19:53.679489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3e1f3-0111-4e1b-bbb3-f394cacc41e5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:53.689416 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 3/4] 10.222.0.57 () {58 vars in 1324 bytes} [Thu Jul 21 17:19:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 17:19:53.695425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:53.710867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c88cfd-2fd3-4bcc-83df-1ae464b43367 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:53.715185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d342-d359-4d9a-b7e6-67bfb4c37199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:53.717313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d342-d359-4d9a-b7e6-67bfb4c37199 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:53.717601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d342-d359-4d9a-b7e6-67bfb4c37199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:53.717877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d342-d359-4d9a-b7e6-67bfb4c37199 admin admin] PATH_INFO: `/v3/projects/ee47bf64e42844579243da3bda418cc7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:53.719396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572d342-d359-4d9a-b7e6-67bfb4c37199 admin admin] RBAC: Authorizing `identity:get_project(project_id=ee47bf64e42844579243da3bda418cc7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:53.774500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5572d342-d359-4d9a-b7e6-67bfb4c37199 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 17:19:53.775124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572d342-d359-4d9a-b7e6-67bfb4c37199 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:53.780617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 2/5] 10.222.0.57 () {56 vars in 1207 bytes} [Thu Jul 21 17:19:53 2022] GET /identity/v3/projects/ee47bf64e42844579243da3bda418cc7 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:19:55.782406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c0bb-8ef4-466d-bf21-1147476fd515 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:55.782957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c0bb-8ef4-466d-bf21-1147476fd515 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:55.782957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c0bb-8ef4-466d-bf21-1147476fd515 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:55.787056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 4/6] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:19:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:19:55.792491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd06f-3249-4ac9-aeb8-36e582c3f266 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:55.792868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd06f-3249-4ac9-aeb8-36e582c3f266 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:55.793046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd06f-3249-4ac9-aeb8-36e582c3f266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:55.812205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-20ecd06f-3249-4ac9-aeb8-36e582c3f266 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:19:55.813740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-20ecd06f-3249-4ac9-aeb8-36e582c3f266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:19:55.821722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 3/7] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:19:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 17:19:55.844344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9c0bb-8ef4-466d-bf21-1147476fd515 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:55.856955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9c0bb-8ef4-466d-bf21-1147476fd515 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:55.861175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce02a3b-7410-4368-ab57-34f5be975e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:55.861812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce02a3b-7410-4368-ab57-34f5be975e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:55.862101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce02a3b-7410-4368-ab57-34f5be975e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:55.862382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce02a3b-7410-4368-ab57-34f5be975e05 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:55.870128 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce02a3b-7410-4368-ab57-34f5be975e05 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:55.871906 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce02a3b-7410-4368-ab57-34f5be975e05 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:55.880707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 5/8] 10.222.0.57 () {58 vars in 1324 bytes} [Thu Jul 21 17:19:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 17:19:55.886111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecd06f-3249-4ac9-aeb8-36e582c3f266 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:55.895522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecd06f-3249-4ac9-aeb8-36e582c3f266 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:55.899817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d847236-2414-4e4d-a542-b546080a42f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:55.900705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d847236-2414-4e4d-a542-b546080a42f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:55.900992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d847236-2414-4e4d-a542-b546080a42f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:55.901270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d847236-2414-4e4d-a542-b546080a42f9 admin admin] PATH_INFO: `/v3/users/57de169446f4409f8d2242abb44a08b7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:55.902652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d847236-2414-4e4d-a542-b546080a42f9 admin admin] RBAC: Authorizing `identity:get_user(user_id=57de169446f4409f8d2242abb44a08b7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:55.905465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d847236-2414-4e4d-a542-b546080a42f9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:55.907024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 4/9] 10.222.0.57 () {56 vars in 1198 bytes} [Thu Jul 21 17:19:55 2022] GET /identity/v3/users/57de169446f4409f8d2242abb44a08b7 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:19:57.979666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b77dad-fae7-4d78-ab40-ba74a1ba205e None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:57.980166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b77dad-fae7-4d78-ab40-ba74a1ba205e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:57.980253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b77dad-fae7-4d78-ab40-ba74a1ba205e None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:57.982169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 6/10] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:19:57.987461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068479e-674d-4fbe-972d-c58d8c774b24 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:57.987829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068479e-674d-4fbe-972d-c58d8c774b24 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:57.988014 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068479e-674d-4fbe-972d-c58d8c774b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.005007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4068479e-674d-4fbe-972d-c58d8c774b24 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:19:58.006176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4068479e-674d-4fbe-972d-c58d8c774b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:19:58.027358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7948-9a92-4d9b-ba26-4f183cb2583f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:58.028367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7948-9a92-4d9b-ba26-4f183cb2583f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:58.028625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7948-9a92-4d9b-ba26-4f183cb2583f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.030040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 7/11] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:19:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:19:58.035308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec335719-a482-46b7-9509-fd5cec378d8f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:58.035601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec335719-a482-46b7-9509-fd5cec378d8f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:58.035885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec335719-a482-46b7-9509-fd5cec378d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.076376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG passlib.handlers.bcrypt [None req-ec335719-a482-46b7-9509-fd5cec378d8f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80211) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 17:19:58.076807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG passlib.handlers.bcrypt [None req-ec335719-a482-46b7-9509-fd5cec378d8f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80211) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 17:19:58.086574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 5/12] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:19:58.109766 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068479e-674d-4fbe-972d-c58d8c774b24 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:58.123262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ec335719-a482-46b7-9509-fd5cec378d8f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:19:58.124882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ec335719-a482-46b7-9509-fd5cec378d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:19:58.132683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:19:58.211853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068479e-674d-4fbe-972d-c58d8c774b24 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:58.223501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 8/13] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:19:58.245861 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec335719-a482-46b7-9509-fd5cec378d8f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:58.246329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbf7ac-3630-44c0-b0a4-77dfe797dcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:58.247150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbf7ac-3630-44c0-b0a4-77dfe797dcc8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:58.247432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbf7ac-3630-44c0-b0a4-77dfe797dcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:58.247707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbf7ac-3630-44c0-b0a4-77dfe797dcc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.258999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbf7ac-3630-44c0-b0a4-77dfe797dcc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-81dbf7ac-3630-44c0-b0a4-77dfe797dcc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:19:58.270749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:19:58.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:19:58.272525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 6/14] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:19:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:19:58.278179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbf7ac-3630-44c0-b0a4-77dfe797dcc8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:58.347702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec335719-a482-46b7-9509-fd5cec378d8f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:58.349997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:19:58.358123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:19:58.376437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbf7ac-3630-44c0-b0a4-77dfe797dcc8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:58.385796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254158f-13f6-45a5-a3b4-51f6682a440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:58.386695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254158f-13f6-45a5-a3b4-51f6682a440d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:58.386988 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254158f-13f6-45a5-a3b4-51f6682a440d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:58.387869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254158f-13f6-45a5-a3b4-51f6682a440d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.410872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540cd22-99be-4d0f-b403-0fac0eafe679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:58.411535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540cd22-99be-4d0f-b403-0fac0eafe679 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:58.411820 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540cd22-99be-4d0f-b403-0fac0eafe679 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:58.412096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540cd22-99be-4d0f-b403-0fac0eafe679 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.412877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540cd22-99be-4d0f-b403-0fac0eafe679 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:19:58.413185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540cd22-99be-4d0f-b403-0fac0eafe679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:58.414573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540cd22-99be-4d0f-b403-0fac0eafe679 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:58.426412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 7/15] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:19:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:19:58.431911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540cd22-99be-4d0f-b403-0fac0eafe679 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:58.456536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254158f-13f6-45a5-a3b4-51f6682a440d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:58.458066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254158f-13f6-45a5-a3b4-51f6682a440d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:58.530033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540cd22-99be-4d0f-b403-0fac0eafe679 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:58.553073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 9/16] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 17:19:58.559037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254158f-13f6-45a5-a3b4-51f6682a440d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:58.561869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:19:58.562955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbb0df-f6c9-4802-9edb-d5e6970347e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:58.564000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbb0df-f6c9-4802-9edb-d5e6970347e9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:58.564324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbb0df-f6c9-4802-9edb-d5e6970347e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:58.564651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbb0df-f6c9-4802-9edb-d5e6970347e9 None admin] PATH_INFO: `/v3/users/57de169446f4409f8d2242abb44a08b7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.566031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbb0df-f6c9-4802-9edb-d5e6970347e9 None admin] RBAC: Authorizing `identity:get_user(user_id=57de169446f4409f8d2242abb44a08b7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:58.567486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbb0df-f6c9-4802-9edb-d5e6970347e9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:58.569128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 8/17] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:19:58 2022] GET /identity/v3/users/57de169446f4409f8d2242abb44a08b7 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:19:58.575195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbb0df-f6c9-4802-9edb-d5e6970347e9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:58.589297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:19:58.658241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254158f-13f6-45a5-a3b4-51f6682a440d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:58.674835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbb0df-f6c9-4802-9edb-d5e6970347e9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:58.691676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c348d2-48f5-4295-9096-64f865dea399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:58.692492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c348d2-48f5-4295-9096-64f865dea399 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:58.692775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c348d2-48f5-4295-9096-64f865dea399 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:58.693045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c348d2-48f5-4295-9096-64f865dea399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.708574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bc810-41cc-4005-875c-b5ac81ba58a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:58.709365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bc810-41cc-4005-875c-b5ac81ba58a0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:58.709683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bc810-41cc-4005-875c-b5ac81ba58a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:58.709982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bc810-41cc-4005-875c-b5ac81ba58a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.711253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bc810-41cc-4005-875c-b5ac81ba58a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:58.711843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c348d2-48f5-4295-9096-64f865dea399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:58.712832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bc810-41cc-4005-875c-b5ac81ba58a0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:58.713320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c348d2-48f5-4295-9096-64f865dea399 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:58.714684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:19:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:19:58.721086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bc810-41cc-4005-875c-b5ac81ba58a0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-06c348d2-48f5-4295-9096-64f865dea399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:19:58.736213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:19:58.737108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:19:58.737893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:19:58.737893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:19:58.737893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:19:58.737893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:19:58.737893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:19:58.743248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c348d2-48f5-4295-9096-64f865dea399 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:19:58.798963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:19:58.804868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:19:58.824013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bc810-41cc-4005-875c-b5ac81ba58a0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:58.844971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c348d2-48f5-4295-9096-64f865dea399 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:19:58.858269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2384ad-73ee-4948-9c78-1532b389f6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:58.858933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2384ad-73ee-4948-9c78-1532b389f6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:58.859327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2384ad-73ee-4948-9c78-1532b389f6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:58.859614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2384ad-73ee-4948-9c78-1532b389f6a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.860472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2384ad-73ee-4948-9c78-1532b389f6a5 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.domain.id=default, user.id=57de169446f4409f8d2242abb44a08b7) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:19:58.860791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2384ad-73ee-4948-9c78-1532b389f6a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:58.862240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2384ad-73ee-4948-9c78-1532b389f6a5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:58.868913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 10/20] 10.222.0.57 () {56 vars in 1322 bytes} [Thu Jul 21 17:19:58 2022] GET /identity/v3/role_assignments?user.id=57de169446f4409f8d2242abb44a08b7&scope.domain.id=default&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:19:58.879983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035a41e-3482-443a-9e88-91be0bef59df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:19:58.880707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035a41e-3482-443a-9e88-91be0bef59df None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:19:58.880990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035a41e-3482-443a-9e88-91be0bef59df None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:19:58.881263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035a41e-3482-443a-9e88-91be0bef59df None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:19:58.882011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035a41e-3482-443a-9e88-91be0bef59df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:19:58.882316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035a41e-3482-443a-9e88-91be0bef59df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:19:58.883767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035a41e-3482-443a-9e88-91be0bef59df None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:19:58.903991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 11/21] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:01.456289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475cc57-f1b1-4b72-bf6a-fb4bdced077a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:01.456842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475cc57-f1b1-4b72-bf6a-fb4bdced077a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:01.456842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475cc57-f1b1-4b72-bf6a-fb4bdced077a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:01.458230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 11/22] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:01.474264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4734a-33c9-4741-bbcf-7f5a920ff531 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:01.474740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4734a-33c9-4741-bbcf-7f5a920ff531 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:01.474821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4734a-33c9-4741-bbcf-7f5a920ff531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:01.493390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d220-99a6-452e-812f-d39888acdbaa None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:01.493670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d220-99a6-452e-812f-d39888acdbaa None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:01.493957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d220-99a6-452e-812f-d39888acdbaa None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:01.495463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 12/23] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:01.500102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8ab4734a-33c9-4741-bbcf-7f5a920ff531 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:01.500948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb14ad-dbd8-4a8c-9aaa-3661edf051b5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:01.501235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb14ad-dbd8-4a8c-9aaa-3661edf051b5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:01.501338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8ab4734a-33c9-4741-bbcf-7f5a920ff531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:01.501502 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb14ad-dbd8-4a8c-9aaa-3661edf051b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:01.530916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-91fb14ad-dbd8-4a8c-9aaa-3661edf051b5 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:01.532123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-91fb14ad-dbd8-4a8c-9aaa-3661edf051b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:01.631187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 12/24] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:01.671135 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 13/25] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:01.711973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4734a-33c9-4741-bbcf-7f5a920ff531 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:01.717056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb14ad-dbd8-4a8c-9aaa-3661edf051b5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:01.786814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:01.853630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4734a-33c9-4741-bbcf-7f5a920ff531 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:01.903355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab30c45-bc60-4bed-b7e6-8b698bf2e901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:01.904547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab30c45-bc60-4bed-b7e6-8b698bf2e901 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:01.905154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab30c45-bc60-4bed-b7e6-8b698bf2e901 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:01.905652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab30c45-bc60-4bed-b7e6-8b698bf2e901 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:01.910099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:01.924994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab30c45-bc60-4bed-b7e6-8b698bf2e901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8ab30c45-bc60-4bed-b7e6-8b698bf2e901 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:01.940491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:01.941382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:01.942118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 13/26] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:01.950776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab30c45-bc60-4bed-b7e6-8b698bf2e901 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:02.006341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb14ad-dbd8-4a8c-9aaa-3661edf051b5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:02.063008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec201e7-cfca-4033-9263-c4c8deb15366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:02.063806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec201e7-cfca-4033-9263-c4c8deb15366 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.064083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec201e7-cfca-4033-9263-c4c8deb15366 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.064376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec201e7-cfca-4033-9263-c4c8deb15366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.086232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec201e7-cfca-4033-9263-c4c8deb15366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:02.089296 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec201e7-cfca-4033-9263-c4c8deb15366 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-fec201e7-cfca-4033-9263-c4c8deb15366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:20:02.117312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:02.118165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:02.119391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:20:02.119391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:20:02.119391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:20:02.119391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:02.119391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 14/27] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:02.119780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e948ab-dd82-4d68-a66a-9aba9267fea7 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.120324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e948ab-dd82-4d68-a66a-9aba9267fea7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.120324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e948ab-dd82-4d68-a66a-9aba9267fea7 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.130163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab30c45-bc60-4bed-b7e6-8b698bf2e901 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:02.131888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 15/28] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:02 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:02.132706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e948ab-dd82-4d68-a66a-9aba9267fea7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:02.144357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:02.194383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee85273-647c-408d-8efb-0a7c5ab89dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:02.196463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85273-647c-408d-8efb-0a7c5ab89dfb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.197072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85273-647c-408d-8efb-0a7c5ab89dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.198092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85273-647c-408d-8efb-0a7c5ab89dfb None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.199188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee85273-647c-408d-8efb-0a7c5ab89dfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:02.204070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee85273-647c-408d-8efb-0a7c5ab89dfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:02.205666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee85273-647c-408d-8efb-0a7c5ab89dfb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:02.225247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 14/29] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:02.227991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ff8aa-59f0-4278-9e5e-f40beacd9485 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.232514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ff8aa-59f0-4278-9e5e-f40beacd9485 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.233105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ff8aa-59f0-4278-9e5e-f40beacd9485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.263673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f64ff8aa-59f0-4278-9e5e-f40beacd9485 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:02.265208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f64ff8aa-59f0-4278-9e5e-f40beacd9485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:02.296678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e948ab-dd82-4d68-a66a-9aba9267fea7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:02.323418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:02.334043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d136ec8-7efc-4f26-b90b-eb7f37703d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:02.334856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d136ec8-7efc-4f26-b90b-eb7f37703d2e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.335242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d136ec8-7efc-4f26-b90b-eb7f37703d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.335621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d136ec8-7efc-4f26-b90b-eb7f37703d2e None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.336499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d136ec8-7efc-4f26-b90b-eb7f37703d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:02.336909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d136ec8-7efc-4f26-b90b-eb7f37703d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:02.338409 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d136ec8-7efc-4f26-b90b-eb7f37703d2e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:02.357623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 15/30] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:02.358787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ff8aa-59f0-4278-9e5e-f40beacd9485 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:02.362258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 16/31] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:02.363785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8769995-0cfe-4098-ab81-2448a8cf5fb5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.364151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8769995-0cfe-4098-ab81-2448a8cf5fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.364777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8769995-0cfe-4098-ab81-2448a8cf5fb5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.366193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 17/32] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:02.370819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8769995-0cfe-4098-ab81-2448a8cf5fb5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:02.479907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:02.486107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ff8aa-59f0-4278-9e5e-f40beacd9485 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:02.486340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8769995-0cfe-4098-ab81-2448a8cf5fb5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:02.522204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa1bfd-0c77-40ce-95b6-dbb70a7ff220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:02.522791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa1bfd-0c77-40ce-95b6-dbb70a7ff220 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.523080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa1bfd-0c77-40ce-95b6-dbb70a7ff220 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.523362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa1bfd-0c77-40ce-95b6-dbb70a7ff220 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.524074 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa1bfd-0c77-40ce-95b6-dbb70a7ff220 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:02.525464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad773d35-4720-441e-a9f0-94fb6d8f84c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:02.525581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa1bfd-0c77-40ce-95b6-dbb70a7ff220 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:02.526727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773d35-4720-441e-a9f0-94fb6d8f84c4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.527297 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773d35-4720-441e-a9f0-94fb6d8f84c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.528430 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773d35-4720-441e-a9f0-94fb6d8f84c4 None admin] PATH_INFO: `/v3/users/57de169446f4409f8d2242abb44a08b7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.530366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad773d35-4720-441e-a9f0-94fb6d8f84c4 None admin] RBAC: Authorizing `identity:get_user(user_id=57de169446f4409f8d2242abb44a08b7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:02.532282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad773d35-4720-441e-a9f0-94fb6d8f84c4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:02.534281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 16/33] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:02 2022] GET /identity/v3/users/57de169446f4409f8d2242abb44a08b7 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:02.535784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29647a13-c7b0-4f36-87c3-f34cc4f40267 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.536262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29647a13-c7b0-4f36-87c3-f34cc4f40267 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.536846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29647a13-c7b0-4f36-87c3-f34cc4f40267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.557416 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-29647a13-c7b0-4f36-87c3-f34cc4f40267 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:02.559017 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-29647a13-c7b0-4f36-87c3-f34cc4f40267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:02.637522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 17/34] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:02.638700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29647a13-c7b0-4f36-87c3-f34cc4f40267 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:02.692277 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 18/35] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:20:02 2022] POST /identity/v3/projects => generated 301 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:02.692876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa1bfd-0c77-40ce-95b6-dbb70a7ff220 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:02.755918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:02.768642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:02.782478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29647a13-c7b0-4f36-87c3-f34cc4f40267 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:02.821507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b251a-40c0-4448-82a1-99ae5f4a0451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:02.822199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b251a-40c0-4448-82a1-99ae5f4a0451 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.822487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b251a-40c0-4448-82a1-99ae5f4a0451 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.822770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b251a-40c0-4448-82a1-99ae5f4a0451 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.823544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b251a-40c0-4448-82a1-99ae5f4a0451 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:02.825490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b251a-40c0-4448-82a1-99ae5f4a0451 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:02.832134 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa1bfd-0c77-40ce-95b6-dbb70a7ff220 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:02.870584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dff95-0aad-4f6e-8d9f-53a6d3eac4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:02.871410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dff95-0aad-4f6e-8d9f-53a6d3eac4fc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.871701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dff95-0aad-4f6e-8d9f-53a6d3eac4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.871970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dff95-0aad-4f6e-8d9f-53a6d3eac4fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.873263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dff95-0aad-4f6e-8d9f-53a6d3eac4fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:02.874961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dff95-0aad-4f6e-8d9f-53a6d3eac4fc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:02.876640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 19/36] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:02.877388 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dff95-0aad-4f6e-8d9f-53a6d3eac4fc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:02.952086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 18/37] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Jul 21 17:20:02 2022] POST /identity/v3/roles => generated 225 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:02.953651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0f89a-a3bb-42e2-8d3a-0bde340abe99 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.953950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0f89a-a3bb-42e2-8d3a-0bde340abe99 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.954241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0f89a-a3bb-42e2-8d3a-0bde340abe99 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.955632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 19/38] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:02.959831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951959e-89dc-48d0-8b04-2bd27ef93951 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:02.959831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951959e-89dc-48d0-8b04-2bd27ef93951 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:02.959831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951959e-89dc-48d0-8b04-2bd27ef93951 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:02.959831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 20/39] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:02.959831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951959e-89dc-48d0-8b04-2bd27ef93951 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:03.001656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dff95-0aad-4f6e-8d9f-53a6d3eac4fc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:03.010278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:03.042157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879fe81-a328-4dac-8774-1cad06897ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:03.042747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879fe81-a328-4dac-8774-1cad06897ad8 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:03.043031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879fe81-a328-4dac-8774-1cad06897ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:03.043372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879fe81-a328-4dac-8774-1cad06897ad8 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:03.044140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879fe81-a328-4dac-8774-1cad06897ad8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:03.045590 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879fe81-a328-4dac-8774-1cad06897ad8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:03.062584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 20/40] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Jul 21 17:20:02 2022] POST /identity/v3/roles => generated 219 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:03.063982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef7723-c4bd-4f29-8d60-cca663705532 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:03.064273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef7723-c4bd-4f29-8d60-cca663705532 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:03.064601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef7723-c4bd-4f29-8d60-cca663705532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:03.085928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8cef7723-c4bd-4f29-8d60-cca663705532 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:03.087599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8cef7723-c4bd-4f29-8d60-cca663705532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:03.097997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951959e-89dc-48d0-8b04-2bd27ef93951 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:03.156624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7f835-1a5a-40aa-88e4-2fb8d0a1637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:03.159434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f835-1a5a-40aa-88e4-2fb8d0a1637a None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:03.159434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f835-1a5a-40aa-88e4-2fb8d0a1637a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:03.159434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f835-1a5a-40aa-88e4-2fb8d0a1637a None admin] PATH_INFO: `/v3/domains/default/users/57de169446f4409f8d2242abb44a08b7/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:03.164585 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7f835-1a5a-40aa-88e4-2fb8d0a1637a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=57de169446f4409f8d2242abb44a08b7, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:03.166106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7f835-1a5a-40aa-88e4-2fb8d0a1637a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:03.180877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 21/41] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:03.182496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee83c0-81ca-4730-8b46-c219a4396bae None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:03.182992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee83c0-81ca-4730-8b46-c219a4396bae None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:03.183441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee83c0-81ca-4730-8b46-c219a4396bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:03.185732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 21/42] 10.222.0.57 () {58 vars in 1361 bytes} [Thu Jul 21 17:20:02 2022] PUT /identity/v3/domains/default/users/57de169446f4409f8d2242abb44a08b7/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:03.209314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7f835-1a5a-40aa-88e4-2fb8d0a1637a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:03.217546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-29ee83c0-81ca-4730-8b46-c219a4396bae None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:03.218173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-29ee83c0-81ca-4730-8b46-c219a4396bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:03.239043 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:03.313166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 22/43] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:03.337683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee83c0-81ca-4730-8b46-c219a4396bae None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:03.357717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7f835-1a5a-40aa-88e4-2fb8d0a1637a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:03.370145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:03.397571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb30d-47cf-4b89-8243-f28d7c8e184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:03.398345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb30d-47cf-4b89-8243-f28d7c8e184b None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:03.398642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb30d-47cf-4b89-8243-f28d7c8e184b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:03.398925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb30d-47cf-4b89-8243-f28d7c8e184b None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:03.399848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdb30d-47cf-4b89-8243-f28d7c8e184b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:03.401113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdb30d-47cf-4b89-8243-f28d7c8e184b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:03.416099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 22/44] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Jul 21 17:20:03 2022] POST /identity/v3/roles => generated 223 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:20:03.453176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee83c0-81ca-4730-8b46-c219a4396bae None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:03.491439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a76f8-466d-4459-847d-621e709ff921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:03.492282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a76f8-466d-4459-847d-621e709ff921 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:03.492578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a76f8-466d-4459-847d-621e709ff921 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:03.492871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a76f8-466d-4459-847d-621e709ff921 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:03.494107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a76f8-466d-4459-847d-621e709ff921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:03.495621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a76f8-466d-4459-847d-621e709ff921 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:03.498150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 23/45] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:03.503961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb30d-47cf-4b89-8243-f28d7c8e184b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:03.588703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:03.614258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb30d-47cf-4b89-8243-f28d7c8e184b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:03.650483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3f22e-b8b7-47ed-b027-2cbab81c5d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:03.651165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3f22e-b8b7-47ed-b027-2cbab81c5d6c None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:03.651517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3f22e-b8b7-47ed-b027-2cbab81c5d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:03.651751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3f22e-b8b7-47ed-b027-2cbab81c5d6c None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:03.652538 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3f22e-b8b7-47ed-b027-2cbab81c5d6c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:03.654268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3f22e-b8b7-47ed-b027-2cbab81c5d6c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:03.682117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 23/46] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:20:03 2022] POST /identity/v3/projects => generated 312 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:05.242786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faf15d-d1f2-4606-9356-61d1704dbce6 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:05.243271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faf15d-d1f2-4606-9356-61d1704dbce6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:05.243271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faf15d-d1f2-4606-9356-61d1704dbce6 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:05.244663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 24/47] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:05.250669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326eadf-3e86-4aa6-b2d8-4f0fedc3b5f5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:05.251054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326eadf-3e86-4aa6-b2d8-4f0fedc3b5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:05.251310 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326eadf-3e86-4aa6-b2d8-4f0fedc3b5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:05.268452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d326eadf-3e86-4aa6-b2d8-4f0fedc3b5f5 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:05.269945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d326eadf-3e86-4aa6-b2d8-4f0fedc3b5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:05.335792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 24/48] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:05.358555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faf15d-d1f2-4606-9356-61d1704dbce6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:05.558850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:05.566802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faf15d-d1f2-4606-9356-61d1704dbce6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:05.601412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab75f4-e38b-49e5-90e4-1d39918a51aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:05.602219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab75f4-e38b-49e5-90e4-1d39918a51aa None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:05.602507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab75f4-e38b-49e5-90e4-1d39918a51aa None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:05.602790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab75f4-e38b-49e5-90e4-1d39918a51aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:05.614012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab75f4-e38b-49e5-90e4-1d39918a51aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-79ab75f4-e38b-49e5-90e4-1d39918a51aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:05.621737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:05.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:05.623337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 25/49] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:05.629093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326eadf-3e86-4aa6-b2d8-4f0fedc3b5f5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:05.684742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:05.726495 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326eadf-3e86-4aa6-b2d8-4f0fedc3b5f5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:05.730418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c834d-a1c6-4ddd-b862-929f7dd8384c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:05.731333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c834d-a1c6-4ddd-b862-929f7dd8384c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:05.731624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c834d-a1c6-4ddd-b862-929f7dd8384c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:05.733048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 26/50] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:05.738027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f6ab1-2064-435a-9e7d-474915c43063 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:05.738326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f6ab1-2064-435a-9e7d-474915c43063 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:05.738613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f6ab1-2064-435a-9e7d-474915c43063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:05.755105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8f8f6ab1-2064-435a-9e7d-474915c43063 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:05.756326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8f8f6ab1-2064-435a-9e7d-474915c43063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:05.761218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de924d-6619-43fb-8ba3-caa10bdbb469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:05.761931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de924d-6619-43fb-8ba3-caa10bdbb469 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:05.762227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de924d-6619-43fb-8ba3-caa10bdbb469 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:05.762512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de924d-6619-43fb-8ba3-caa10bdbb469 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:05.763266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de924d-6619-43fb-8ba3-caa10bdbb469 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:05.763589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de924d-6619-43fb-8ba3-caa10bdbb469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:05.764999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de924d-6619-43fb-8ba3-caa10bdbb469 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:05.773190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 25/51] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:05.779181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de924d-6619-43fb-8ba3-caa10bdbb469 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:05.822037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 27/52] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:05.845802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f6ab1-2064-435a-9e7d-474915c43063 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:05.937514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:05.954085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f6ab1-2064-435a-9e7d-474915c43063 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:05.988131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de924d-6619-43fb-8ba3-caa10bdbb469 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:05.988364 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ca2f8-ad7d-4999-a725-80482186a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:05.989095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ca2f8-ad7d-4999-a725-80482186a477 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:05.989376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ca2f8-ad7d-4999-a725-80482186a477 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:05.989651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ca2f8-ad7d-4999-a725-80482186a477 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:05.990864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ca2f8-ad7d-4999-a725-80482186a477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:05.992353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ca2f8-ad7d-4999-a725-80482186a477 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:05.993957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 28/53] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:05.999645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ca2f8-ad7d-4999-a725-80482186a477 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:06.008999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:06.024272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2745cb-f950-48a9-a7e4-73341e82dad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:06.025177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2745cb-f950-48a9-a7e4-73341e82dad3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:06.025460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2745cb-f950-48a9-a7e4-73341e82dad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:06.025739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2745cb-f950-48a9-a7e4-73341e82dad3 None admin] PATH_INFO: `/v3/users/57de169446f4409f8d2242abb44a08b7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:06.026938 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2745cb-f950-48a9-a7e4-73341e82dad3 None admin] RBAC: Authorizing `identity:get_user(user_id=57de169446f4409f8d2242abb44a08b7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:06.029126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2745cb-f950-48a9-a7e4-73341e82dad3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:06.030784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 26/54] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:05 2022] GET /identity/v3/users/57de169446f4409f8d2242abb44a08b7 => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:06.036050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2745cb-f950-48a9-a7e4-73341e82dad3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:06.103341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ca2f8-ad7d-4999-a725-80482186a477 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:06.134270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2745cb-f950-48a9-a7e4-73341e82dad3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:06.136475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ebe08-76c4-4474-bd8b-af41103f56b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:06.138085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ebe08-76c4-4474-bd8b-af41103f56b9 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:06.138371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ebe08-76c4-4474-bd8b-af41103f56b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:06.138652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ebe08-76c4-4474-bd8b-af41103f56b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:06.139365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ebe08-76c4-4474-bd8b-af41103f56b9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:06.140854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ebe08-76c4-4474-bd8b-af41103f56b9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:06.167025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 29/55] 10.222.0.57 () {60 vars in 1140 bytes} [Thu Jul 21 17:20:05 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:20:06.170400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1446ca9-0ede-4d78-b85e-c6f5d861b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:06.171385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1446ca9-0ede-4d78-b85e-c6f5d861b459 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:06.171777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1446ca9-0ede-4d78-b85e-c6f5d861b459 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:06.172182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1446ca9-0ede-4d78-b85e-c6f5d861b459 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:06.173621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1446ca9-0ede-4d78-b85e-c6f5d861b459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:06.175189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1446ca9-0ede-4d78-b85e-c6f5d861b459 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:06.176935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 27/56] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:06.183573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ebe08-76c4-4474-bd8b-af41103f56b9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:06.187821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:06.292492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ebe08-76c4-4474-bd8b-af41103f56b9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:06.328563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83533eab-242e-47df-8b24-c441db2b2063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:06.329223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83533eab-242e-47df-8b24-c441db2b2063 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:06.329508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83533eab-242e-47df-8b24-c441db2b2063 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:06.329786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83533eab-242e-47df-8b24-c441db2b2063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:06.330622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83533eab-242e-47df-8b24-c441db2b2063 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.domain.id=default, user.id=57de169446f4409f8d2242abb44a08b7) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:06.330935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83533eab-242e-47df-8b24-c441db2b2063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:06.332402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83533eab-242e-47df-8b24-c441db2b2063 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:06.338238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 30/57] 10.222.0.57 () {56 vars in 1322 bytes} [Thu Jul 21 17:20:06 2022] GET /identity/v3/role_assignments?user.id=57de169446f4409f8d2242abb44a08b7&scope.domain.id=default&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:08.215691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df95898-66bc-403c-b005-ec35040ef1db None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:08.216177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df95898-66bc-403c-b005-ec35040ef1db None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:08.216177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df95898-66bc-403c-b005-ec35040ef1db None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:08.217577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 28/58] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:08.222766 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf81a3-4663-41c9-b839-91b8097b65b1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:08.223169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf81a3-4663-41c9-b839-91b8097b65b1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:08.223341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf81a3-4663-41c9-b839-91b8097b65b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:08.240337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-dbcf81a3-4663-41c9-b839-91b8097b65b1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:08.241530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dbcf81a3-4663-41c9-b839-91b8097b65b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:08.301753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:08.307549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 31/59] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:08.332686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df95898-66bc-403c-b005-ec35040ef1db None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:08.388924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:08.431584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df95898-66bc-403c-b005-ec35040ef1db None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:08.463785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81d40c-a7d4-437f-ba77-13f892df4d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:08.464589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81d40c-a7d4-437f-ba77-13f892df4d2d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:08.464874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81d40c-a7d4-437f-ba77-13f892df4d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:08.465371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81d40c-a7d4-437f-ba77-13f892df4d2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:08.466378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81d40c-a7d4-437f-ba77-13f892df4d2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:08.467744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81d40c-a7d4-437f-ba77-13f892df4d2d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:08.469356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 29/60] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:08.474759 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf81a3-4663-41c9-b839-91b8097b65b1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:08.571330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf81a3-4663-41c9-b839-91b8097b65b1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:08.606725 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e694ea8-66d2-4b30-8d1d-409d1f9de0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:08.607343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e694ea8-66d2-4b30-8d1d-409d1f9de0f8 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:08.607628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e694ea8-66d2-4b30-8d1d-409d1f9de0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:08.607906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e694ea8-66d2-4b30-8d1d-409d1f9de0f8 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:08.608643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e694ea8-66d2-4b30-8d1d-409d1f9de0f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:08.610095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e694ea8-66d2-4b30-8d1d-409d1f9de0f8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:08.633995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 32/61] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:20:08 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:11.765385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf2f79-a854-4900-817f-2c8bfa4b6801 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:11.766408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf2f79-a854-4900-817f-2c8bfa4b6801 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:11.766891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf2f79-a854-4900-817f-2c8bfa4b6801 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:11.768749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 30/62] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:11.773798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3a390-69f2-41f3-b868-a32f25db6c3c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:11.774192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3a390-69f2-41f3-b868-a32f25db6c3c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:11.774414 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3a390-69f2-41f3-b868-a32f25db6c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:11.798670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-eba3a390-69f2-41f3-b868-a32f25db6c3c None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:11.800129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-eba3a390-69f2-41f3-b868-a32f25db6c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:11.861921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:11.885311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 33/63] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:11.922111 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf2f79-a854-4900-817f-2c8bfa4b6801 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:11.967319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04b0ff-d21c-4632-92e6-1016c58812be None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:11.968105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04b0ff-d21c-4632-92e6-1016c58812be None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:11.968610 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04b0ff-d21c-4632-92e6-1016c58812be None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:11.971813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 34/64] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:11.976606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e516d-5946-47fe-bb9e-1f3c4bf17ae3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:11.977132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e516d-5946-47fe-bb9e-1f3c4bf17ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:11.977607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e516d-5946-47fe-bb9e-1f3c4bf17ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.002983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a63e516d-5946-47fe-bb9e-1f3c4bf17ae3 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:12.005257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a63e516d-5946-47fe-bb9e-1f3c4bf17ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:12.106867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf2f79-a854-4900-817f-2c8bfa4b6801 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:12.119979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 35/65] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:12.128584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253b08-9825-4503-ba17-1fe75c6a5be8 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.129106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253b08-9825-4503-ba17-1fe75c6a5be8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.129587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253b08-9825-4503-ba17-1fe75c6a5be8 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.131216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 36/66] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:12.139877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff30cfe-dbde-4606-b270-d807c49ad576 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.140472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff30cfe-dbde-4606-b270-d807c49ad576 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.140953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff30cfe-dbde-4606-b270-d807c49ad576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.146077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:12.157204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75755d36-cb0c-42b4-8b1b-85ff86108b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:12.158303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75755d36-cb0c-42b4-8b1b-85ff86108b51 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.158782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75755d36-cb0c-42b4-8b1b-85ff86108b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.159284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75755d36-cb0c-42b4-8b1b-85ff86108b51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.165908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7ff30cfe-dbde-4606-b270-d807c49ad576 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:12.167441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7ff30cfe-dbde-4606-b270-d807c49ad576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:12.171794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75755d36-cb0c-42b4-8b1b-85ff86108b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-75755d36-cb0c-42b4-8b1b-85ff86108b51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:12.180045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:12.181062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:12.181818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:12.182632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 31/67] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:12.183392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75755d36-cb0c-42b4-8b1b-85ff86108b51 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:12.244093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 37/68] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:12.245347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff30cfe-dbde-4606-b270-d807c49ad576 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:12.305759 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:12.325225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75755d36-cb0c-42b4-8b1b-85ff86108b51 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:12.361435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff30cfe-dbde-4606-b270-d807c49ad576 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:12.367372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6844b6-3871-466c-8f33-0e8222d9ffc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:12.368179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6844b6-3871-466c-8f33-0e8222d9ffc6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.368494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6844b6-3871-466c-8f33-0e8222d9ffc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.369190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6844b6-3871-466c-8f33-0e8222d9ffc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.382205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6844b6-3871-466c-8f33-0e8222d9ffc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-9b6844b6-3871-466c-8f33-0e8222d9ffc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:12.392504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:12.393349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:12.394095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 32/69] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:12.394717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6844b6-3871-466c-8f33-0e8222d9ffc6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:12.397632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72552a81-e429-474c-a71c-dabc56b79c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:12.398312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552a81-e429-474c-a71c-dabc56b79c46 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.398594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552a81-e429-474c-a71c-dabc56b79c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.398869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552a81-e429-474c-a71c-dabc56b79c46 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.399624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72552a81-e429-474c-a71c-dabc56b79c46 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:12.400127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72552a81-e429-474c-a71c-dabc56b79c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:12.401568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72552a81-e429-474c-a71c-dabc56b79c46 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:12.410789 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 38/70] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:12.412015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872364e-babe-4d30-9f89-57383147648c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.412321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872364e-babe-4d30-9f89-57383147648c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.412611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872364e-babe-4d30-9f89-57383147648c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.413828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 39/71] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:12.414884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f266ff-3d9a-4820-9d68-8fbc7739ad65 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.415487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f266ff-3d9a-4820-9d68-8fbc7739ad65 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.415487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f266ff-3d9a-4820-9d68-8fbc7739ad65 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.416786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 40/72] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:12.417335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f266ff-3d9a-4820-9d68-8fbc7739ad65 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:12.449652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:12.502617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6844b6-3871-466c-8f33-0e8222d9ffc6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:12.522277 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f266ff-3d9a-4820-9d68-8fbc7739ad65 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:12.530236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:12.538049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce9e50-2e20-45bf-b47b-f4d95a693602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:12.538862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce9e50-2e20-45bf-b47b-f4d95a693602 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.539204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce9e50-2e20-45bf-b47b-f4d95a693602 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.539483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce9e50-2e20-45bf-b47b-f4d95a693602 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.550006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce9e50-2e20-45bf-b47b-f4d95a693602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-25ce9e50-2e20-45bf-b47b-f4d95a693602 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:12.558234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:12.559695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:12.560786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:12.561376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4eef45-1f89-499d-b8fe-9bb05ad75ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:12.561376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4eef45-1f89-499d-b8fe-9bb05ad75ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.561376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4eef45-1f89-499d-b8fe-9bb05ad75ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.561376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4eef45-1f89-499d-b8fe-9bb05ad75ea0 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.561620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 33/73] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:12.561684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4eef45-1f89-499d-b8fe-9bb05ad75ea0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:12.561746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce9e50-2e20-45bf-b47b-f4d95a693602 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:12.561807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4eef45-1f89-499d-b8fe-9bb05ad75ea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:12.562019 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4eef45-1f89-499d-b8fe-9bb05ad75ea0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:12.570189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 41/74] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:12.571472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a61174-ff39-4d20-9ad3-d59853bbcfce None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.571765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a61174-ff39-4d20-9ad3-d59853bbcfce None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.572049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a61174-ff39-4d20-9ad3-d59853bbcfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.587863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-76a61174-ff39-4d20-9ad3-d59853bbcfce None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:12.589066 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-76a61174-ff39-4d20-9ad3-d59853bbcfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:12.655206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 42/75] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:12.656570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac01a52-8f68-4c50-a77a-26ab961bcebf None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.656868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac01a52-8f68-4c50-a77a-26ab961bcebf None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.657150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac01a52-8f68-4c50-a77a-26ab961bcebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.662905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce9e50-2e20-45bf-b47b-f4d95a693602 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:12.665246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:12.683409 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-cac01a52-8f68-4c50-a77a-26ab961bcebf None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:12.684754 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cac01a52-8f68-4c50-a77a-26ab961bcebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:12.697626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b1e33-521f-4cf7-a393-5c525e7d4841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:12.698502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b1e33-521f-4cf7-a393-5c525e7d4841 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.698792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b1e33-521f-4cf7-a393-5c525e7d4841 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.699071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b1e33-521f-4cf7-a393-5c525e7d4841 None admin] PATH_INFO: `/v3/users/f2817f9598a341b1bc87577b4bc383f1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.714652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b1e33-521f-4cf7-a393-5c525e7d4841 None admin] RBAC: Authorizing `identity:get_user(user_id=f2817f9598a341b1bc87577b4bc383f1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:12.716691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b1e33-521f-4cf7-a393-5c525e7d4841 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:12.718372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 34/76] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:12 2022] GET /identity/v3/users/f2817f9598a341b1bc87577b4bc383f1 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:12.719168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b1e33-521f-4cf7-a393-5c525e7d4841 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:12.751753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 43/77] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:12.753231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac01a52-8f68-4c50-a77a-26ab961bcebf None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:12.785558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:12.819232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b1e33-521f-4cf7-a393-5c525e7d4841 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:12.853967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adfb63-4941-4143-9412-492d7940dff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:12.854641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adfb63-4941-4143-9412-492d7940dff2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.855125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adfb63-4941-4143-9412-492d7940dff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.855636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adfb63-4941-4143-9412-492d7940dff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.856909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adfb63-4941-4143-9412-492d7940dff2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:12.857399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adfb63-4941-4143-9412-492d7940dff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:12.858401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac01a52-8f68-4c50-a77a-26ab961bcebf None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:12.859562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adfb63-4941-4143-9412-492d7940dff2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:12.868156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 35/78] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:12.869381 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adfb63-4941-4143-9412-492d7940dff2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:12.894774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:12.896663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba800c-9701-4338-b9d8-765e70c670b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:12.897619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba800c-9701-4338-b9d8-765e70c670b6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:12.897935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba800c-9701-4338-b9d8-765e70c670b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:12.898219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba800c-9701-4338-b9d8-765e70c670b6 None admin] PATH_INFO: `/v3/users/57de169446f4409f8d2242abb44a08b7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:12.899423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba800c-9701-4338-b9d8-765e70c670b6 None admin] RBAC: Authorizing `identity:get_user(user_id=57de169446f4409f8d2242abb44a08b7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:12.900920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba800c-9701-4338-b9d8-765e70c670b6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:12.903208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 44/79] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:12 2022] GET /identity/v3/users/57de169446f4409f8d2242abb44a08b7 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:12.903980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba800c-9701-4338-b9d8-765e70c670b6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:12.971808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adfb63-4941-4143-9412-492d7940dff2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:12.996030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:13.002716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba800c-9701-4338-b9d8-765e70c670b6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:13.004821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972c151-0907-4378-ae19-da43668c5774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:13.005623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972c151-0907-4378-ae19-da43668c5774 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:13.005912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972c151-0907-4378-ae19-da43668c5774 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:13.006194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972c151-0907-4378-ae19-da43668c5774 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:13.017087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972c151-0907-4378-ae19-da43668c5774 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-3972c151-0907-4378-ae19-da43668c5774 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:13.024857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:13.025683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:13.026440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 36/80] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:13.027051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972c151-0907-4378-ae19-da43668c5774 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:13.044013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaba30-c108-42b9-a70b-a76153136784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:13.044722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaba30-c108-42b9-a70b-a76153136784 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:13.045012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaba30-c108-42b9-a70b-a76153136784 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:13.045296 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaba30-c108-42b9-a70b-a76153136784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:13.111552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:13.113191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aaba30-c108-42b9-a70b-a76153136784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:13.114686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aaba30-c108-42b9-a70b-a76153136784 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:13.138431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972c151-0907-4378-ae19-da43668c5774 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:13.170758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63df733-ae77-414d-9c5a-2fd520472786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:13.171548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63df733-ae77-414d-9c5a-2fd520472786 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:13.171835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63df733-ae77-414d-9c5a-2fd520472786 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:13.172202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63df733-ae77-414d-9c5a-2fd520472786 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:13.173361 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63df733-ae77-414d-9c5a-2fd520472786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:13.174834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63df733-ae77-414d-9c5a-2fd520472786 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:13.176466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 37/81] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:13.177235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63df733-ae77-414d-9c5a-2fd520472786 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:13.210096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:13.217929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 45/82] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:13.218749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaba30-c108-42b9-a70b-a76153136784 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:13.278619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63df733-ae77-414d-9c5a-2fd520472786 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:13.311931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e573a-445d-44ea-bd19-4310a0fe7405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:13.312837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e573a-445d-44ea-bd19-4310a0fe7405 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:13.313124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e573a-445d-44ea-bd19-4310a0fe7405 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:13.313407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e573a-445d-44ea-bd19-4310a0fe7405 None admin] PATH_INFO: `/v3/users/f2817f9598a341b1bc87577b4bc383f1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:13.314629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e573a-445d-44ea-bd19-4310a0fe7405 None admin] RBAC: Authorizing `identity:get_user(user_id=f2817f9598a341b1bc87577b4bc383f1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:13.315996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaba30-c108-42b9-a70b-a76153136784 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:13.316104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e573a-445d-44ea-bd19-4310a0fe7405 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:13.317726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 38/83] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:13 2022] GET /identity/v3/users/f2817f9598a341b1bc87577b4bc383f1 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:13.318496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e573a-445d-44ea-bd19-4310a0fe7405 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:13.341288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:13.355193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9843e-d1e9-4a02-93cc-8d9f3b85ab28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:13.356715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9843e-d1e9-4a02-93cc-8d9f3b85ab28 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:13.357009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9843e-d1e9-4a02-93cc-8d9f3b85ab28 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:13.357289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9843e-d1e9-4a02-93cc-8d9f3b85ab28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:13.417825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:13.421712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e573a-445d-44ea-bd19-4310a0fe7405 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:13.425200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9843e-d1e9-4a02-93cc-8d9f3b85ab28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:13.426678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9843e-d1e9-4a02-93cc-8d9f3b85ab28 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:13.455513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a0adb-b3d5-4d40-ad76-269a6f8d9f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:13.456182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a0adb-b3d5-4d40-ad76-269a6f8d9f9e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:13.456500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a0adb-b3d5-4d40-ad76-269a6f8d9f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:13.456785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a0adb-b3d5-4d40-ad76-269a6f8d9f9e None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:13.457529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a0adb-b3d5-4d40-ad76-269a6f8d9f9e None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:13.457839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a0adb-b3d5-4d40-ad76-269a6f8d9f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:13.459198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a0adb-b3d5-4d40-ad76-269a6f8d9f9e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:13.467808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 39/84] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:13.468546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a0adb-b3d5-4d40-ad76-269a6f8d9f9e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:13.523995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 46/85] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:13.524945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9843e-d1e9-4a02-93cc-8d9f3b85ab28 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:13.550497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:13.568652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a0adb-b3d5-4d40-ad76-269a6f8d9f9e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:13.601638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc41dd6-5236-467a-a1b7-554f908335c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:13.602301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc41dd6-5236-467a-a1b7-554f908335c5 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:13.602586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc41dd6-5236-467a-a1b7-554f908335c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:13.602862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc41dd6-5236-467a-a1b7-554f908335c5 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:13.603603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc41dd6-5236-467a-a1b7-554f908335c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:13.605059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc41dd6-5236-467a-a1b7-554f908335c5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:13.619739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:13.627144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9843e-d1e9-4a02-93cc-8d9f3b85ab28 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:13.630650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 40/86] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:20:13 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:13.631469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc41dd6-5236-467a-a1b7-554f908335c5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:13.669181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a508652-7aa6-4c7c-ad6a-0e09d56c5400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:13.670070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508652-7aa6-4c7c-ad6a-0e09d56c5400 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:13.670354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508652-7aa6-4c7c-ad6a-0e09d56c5400 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:13.670630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508652-7aa6-4c7c-ad6a-0e09d56c5400 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:13.672693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a508652-7aa6-4c7c-ad6a-0e09d56c5400 None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:13.673375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a508652-7aa6-4c7c-ad6a-0e09d56c5400 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:13.680088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 47/87] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:13 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:13.680088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a508652-7aa6-4c7c-ad6a-0e09d56c5400 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:13.749059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc41dd6-5236-467a-a1b7-554f908335c5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:13.787247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836316c-d016-4b08-b789-8c41ea61845f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:13.787907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836316c-d016-4b08-b789-8c41ea61845f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:13.788253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836316c-d016-4b08-b789-8c41ea61845f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:13.788619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836316c-d016-4b08-b789-8c41ea61845f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:13.796486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a508652-7aa6-4c7c-ad6a-0e09d56c5400 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:13.798162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:13.844496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d2fdb-0a74-4c65-94f5-91817e01b011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:13.845393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d2fdb-0a74-4c65-94f5-91817e01b011 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:13.845988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d2fdb-0a74-4c65-94f5-91817e01b011 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:13.845988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d2fdb-0a74-4c65-94f5-91817e01b011 None admin] PATH_INFO: `/v3/users/f2817f9598a341b1bc87577b4bc383f1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:13.848926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d2fdb-0a74-4c65-94f5-91817e01b011 None admin] RBAC: Authorizing `identity:get_user(user_id=f2817f9598a341b1bc87577b4bc383f1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:13.851609 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d2fdb-0a74-4c65-94f5-91817e01b011 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:13.855307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 48/88] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:13 2022] GET /identity/v3/users/f2817f9598a341b1bc87577b4bc383f1 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:13.855307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d2fdb-0a74-4c65-94f5-91817e01b011 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:13.876055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836316c-d016-4b08-b789-8c41ea61845f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:13.877637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836316c-d016-4b08-b789-8c41ea61845f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:13.900568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:13.993283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d2fdb-0a74-4c65-94f5-91817e01b011 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:14.007191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 41/89] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:14.008257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836316c-d016-4b08-b789-8c41ea61845f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:14.043229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ec15a-17b0-4443-9ab0-acc88ee869cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:14.044588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ec15a-17b0-4443-9ab0-acc88ee869cc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:14.044982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ec15a-17b0-4443-9ab0-acc88ee869cc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:14.045371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ec15a-17b0-4443-9ab0-acc88ee869cc None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:14.047365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ec15a-17b0-4443-9ab0-acc88ee869cc None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:14.049296 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ec15a-17b0-4443-9ab0-acc88ee869cc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:14.052350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 49/90] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:13 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:14.057814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ec15a-17b0-4443-9ab0-acc88ee869cc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:14.062652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:14.129768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836316c-d016-4b08-b789-8c41ea61845f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:14.169594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264da8c-e090-41f1-bd34-c80265cbe2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:14.170615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264da8c-e090-41f1-bd34-c80265cbe2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:14.171097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264da8c-e090-41f1-bd34-c80265cbe2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:14.171645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264da8c-e090-41f1-bd34-c80265cbe2b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:14.172751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264da8c-e090-41f1-bd34-c80265cbe2b9 None admin] RBAC: Adding query filter params (role.id=d2c2845de1d548bb9cfb46f70619b3f8, scope.project.id=380cd1f0793045d280c092a275d9666a, user.id=f2817f9598a341b1bc87577b4bc383f1) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:14.173276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264da8c-e090-41f1-bd34-c80265cbe2b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:14.174924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264da8c-e090-41f1-bd34-c80265cbe2b9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:14.185570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 42/91] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:14 2022] GET /identity/v3/role_assignments?user.id=f2817f9598a341b1bc87577b4bc383f1&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:14.192936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:14.193743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264da8c-e090-41f1-bd34-c80265cbe2b9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:14.196756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ec15a-17b0-4443-9ab0-acc88ee869cc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:14.244345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ffd81-2cbb-4bb6-9b24-e416f5a8387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:14.245083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ffd81-2cbb-4bb6-9b24-e416f5a8387b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:14.245374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ffd81-2cbb-4bb6-9b24-e416f5a8387b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:14.245658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ffd81-2cbb-4bb6-9b24-e416f5a8387b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:14.335006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ffd81-2cbb-4bb6-9b24-e416f5a8387b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:14.336440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ffd81-2cbb-4bb6-9b24-e416f5a8387b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:14.430768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264da8c-e090-41f1-bd34-c80265cbe2b9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:14.432897 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:14.466430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:14.471844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 50/92] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:14.472795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ffd81-2cbb-4bb6-9b24-e416f5a8387b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:14.474012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b54f6-c02c-4f46-981d-7d9fcb28ee2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:14.475109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b54f6-c02c-4f46-981d-7d9fcb28ee2e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:14.475586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b54f6-c02c-4f46-981d-7d9fcb28ee2e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:14.476070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b54f6-c02c-4f46-981d-7d9fcb28ee2e None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:14.477547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b54f6-c02c-4f46-981d-7d9fcb28ee2e None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:14.479235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b54f6-c02c-4f46-981d-7d9fcb28ee2e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:14.481787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 43/93] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:14 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:14.482788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b54f6-c02c-4f46-981d-7d9fcb28ee2e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:14.598217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ffd81-2cbb-4bb6-9b24-e416f5a8387b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:14.647893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b54f6-c02c-4f46-981d-7d9fcb28ee2e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:14.666772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08685040-a532-4d25-84b6-32922e007402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:14.667445 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08685040-a532-4d25-84b6-32922e007402 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:14.667733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08685040-a532-4d25-84b6-32922e007402 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:14.668052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08685040-a532-4d25-84b6-32922e007402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:14.668926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08685040-a532-4d25-84b6-32922e007402 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=380cd1f0793045d280c092a275d9666a, user.id=57de169446f4409f8d2242abb44a08b7) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:14.669246 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08685040-a532-4d25-84b6-32922e007402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:14.670703 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08685040-a532-4d25-84b6-32922e007402 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:14.685032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 51/94] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:14 2022] GET /identity/v3/role_assignments?user.id=57de169446f4409f8d2242abb44a08b7&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:14.685886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08685040-a532-4d25-84b6-32922e007402 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:14.720859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff7f83-4dac-45f4-af3d-3c56a269a7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:14.722921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff7f83-4dac-45f4-af3d-3c56a269a7aa None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:14.723231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff7f83-4dac-45f4-af3d-3c56a269a7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:14.723519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff7f83-4dac-45f4-af3d-3c56a269a7aa None admin] PATH_INFO: `/v3/projects/cba3324a947d41dc92892d0fe15c84f9` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:14.727680 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff7f83-4dac-45f4-af3d-3c56a269a7aa None admin] RBAC: Authorizing `identity:get_project(project_id=cba3324a947d41dc92892d0fe15c84f9)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:14.730247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff7f83-4dac-45f4-af3d-3c56a269a7aa None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:14.734780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 44/95] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:14 2022] GET /identity/v3/projects/cba3324a947d41dc92892d0fe15c84f9 => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:14.740583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff7f83-4dac-45f4-af3d-3c56a269a7aa None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:14.811406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:14.821466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:14.936421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08685040-a532-4d25-84b6-32922e007402 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:14.974523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff7f83-4dac-45f4-af3d-3c56a269a7aa None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:15.022055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f960d-53ce-4ec3-9fe3-ab57f211f276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:15.022738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f960d-53ce-4ec3-9fe3-ab57f211f276 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:15.023034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f960d-53ce-4ec3-9fe3-ab57f211f276 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:15.023323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f960d-53ce-4ec3-9fe3-ab57f211f276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:15.024172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f960d-53ce-4ec3-9fe3-ab57f211f276 None admin] RBAC: Adding query filter params (role.id=110c20a9ff1e455d8c43ffdfbdd1447b, scope.project.id=380cd1f0793045d280c092a275d9666a, user.id=f2817f9598a341b1bc87577b4bc383f1) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:15.024628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f960d-53ce-4ec3-9fe3-ab57f211f276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:15.026221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f960d-53ce-4ec3-9fe3-ab57f211f276 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:15.038178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b106b09-2586-49d3-b232-3acff78fb3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:15.039132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b106b09-2586-49d3-b232-3acff78fb3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:15.039376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 52/96] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:14 2022] GET /identity/v3/role_assignments?user.id=f2817f9598a341b1bc87577b4bc383f1&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=110c20a9ff1e455d8c43ffdfbdd1447b => generated 259 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:15.039812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b106b09-2586-49d3-b232-3acff78fb3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:15.040284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b106b09-2586-49d3-b232-3acff78fb3c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:15.041773 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b106b09-2586-49d3-b232-3acff78fb3c1 None admin] RBAC: Adding query filter params (role.id=d2c2845de1d548bb9cfb46f70619b3f8, scope.project.id=cba3324a947d41dc92892d0fe15c84f9, user.id=f2817f9598a341b1bc87577b4bc383f1) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:15.042292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b106b09-2586-49d3-b232-3acff78fb3c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:15.044951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b106b09-2586-49d3-b232-3acff78fb3c1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:15.057108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 45/97] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:14 2022] GET /identity/v3/role_assignments?user.id=f2817f9598a341b1bc87577b4bc383f1&scope.project.id=cba3324a947d41dc92892d0fe15c84f9&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 259 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:16.120149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3c47-a6b3-4da7-b0e8-f88edcfb0c1d None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:16.120666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3c47-a6b3-4da7-b0e8-f88edcfb0c1d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:16.120737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3c47-a6b3-4da7-b0e8-f88edcfb0c1d None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:16.123016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 53/98] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:16.128220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e13dff-eb13-4f85-9c0b-d232a6e9f5cf None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:16.129073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e13dff-eb13-4f85-9c0b-d232a6e9f5cf None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:16.129547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e13dff-eb13-4f85-9c0b-d232a6e9f5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:16.153400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-84e13dff-eb13-4f85-9c0b-d232a6e9f5cf None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:16.154892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-84e13dff-eb13-4f85-9c0b-d232a6e9f5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:16.234070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 46/99] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:16.257409 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3c47-a6b3-4da7-b0e8-f88edcfb0c1d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:16.390892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3c47-a6b3-4da7-b0e8-f88edcfb0c1d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:16.423902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:16.432689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac8464-9a18-4495-b5e6-590356e1e161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:16.433634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac8464-9a18-4495-b5e6-590356e1e161 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:16.434873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac8464-9a18-4495-b5e6-590356e1e161 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:16.435164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac8464-9a18-4495-b5e6-590356e1e161 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:16.451639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac8464-9a18-4495-b5e6-590356e1e161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-72ac8464-9a18-4495-b5e6-590356e1e161 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:16.459341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:16.460176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:16.462022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 54/100] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:16.466689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e13dff-eb13-4f85-9c0b-d232a6e9f5cf None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:16.525302 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:16.554320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cf082-481c-4347-b603-ffc7cb8b9330 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:16.554625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cf082-481c-4347-b603-ffc7cb8b9330 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:16.554886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cf082-481c-4347-b603-ffc7cb8b9330 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:16.559210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 55/101] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:16.567911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74994b2b-4d6e-42be-9054-b01cb06ec35f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:16.568206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74994b2b-4d6e-42be-9054-b01cb06ec35f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:16.568508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74994b2b-4d6e-42be-9054-b01cb06ec35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:16.597705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-74994b2b-4d6e-42be-9054-b01cb06ec35f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:16.598920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-74994b2b-4d6e-42be-9054-b01cb06ec35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:16.601368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e13dff-eb13-4f85-9c0b-d232a6e9f5cf None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:16.641863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb2d98-0578-4d6e-8ed5-85dc486f4b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:16.642543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb2d98-0578-4d6e-8ed5-85dc486f4b96 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:16.642826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb2d98-0578-4d6e-8ed5-85dc486f4b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:16.643109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb2d98-0578-4d6e-8ed5-85dc486f4b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:16.643863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb2d98-0578-4d6e-8ed5-85dc486f4b96 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:16.644175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb2d98-0578-4d6e-8ed5-85dc486f4b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:16.645601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb2d98-0578-4d6e-8ed5-85dc486f4b96 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:16.657775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 47/102] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:16.664913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb2d98-0578-4d6e-8ed5-85dc486f4b96 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:16.750452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 56/103] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:16.827918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74994b2b-4d6e-42be-9054-b01cb06ec35f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:16.856000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:16.874460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb2d98-0578-4d6e-8ed5-85dc486f4b96 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:16.965060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeffb42-1b41-4ff0-904c-fac75e69195d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:16.965975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeffb42-1b41-4ff0-904c-fac75e69195d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:16.966267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeffb42-1b41-4ff0-904c-fac75e69195d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:16.966554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeffb42-1b41-4ff0-904c-fac75e69195d None admin] PATH_INFO: `/v3/users/808c0899b68e44a28407207932e719fa` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:16.993033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeffb42-1b41-4ff0-904c-fac75e69195d None admin] RBAC: Authorizing `identity:get_user(user_id=808c0899b68e44a28407207932e719fa)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:16.994555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeffb42-1b41-4ff0-904c-fac75e69195d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:16.996902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 48/104] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:16 2022] GET /identity/v3/users/808c0899b68e44a28407207932e719fa => generated 289 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:17.003113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeffb42-1b41-4ff0-904c-fac75e69195d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:17.054451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74994b2b-4d6e-42be-9054-b01cb06ec35f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:17.087842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:17.128561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c18f-8361-45a4-8fa9-a4d5c8fbd117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:17.129436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c18f-8361-45a4-8fa9-a4d5c8fbd117 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:17.129727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c18f-8361-45a4-8fa9-a4d5c8fbd117 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:17.130016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c18f-8361-45a4-8fa9-a4d5c8fbd117 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:17.132630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39c18f-8361-45a4-8fa9-a4d5c8fbd117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:17.134265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39c18f-8361-45a4-8fa9-a4d5c8fbd117 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:17.136455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 57/105] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:17.142714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c18f-8361-45a4-8fa9-a4d5c8fbd117 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:17.157810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeffb42-1b41-4ff0-904c-fac75e69195d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:17.209654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa0026-7ab3-44ac-aecf-8c37240ff158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:17.210330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa0026-7ab3-44ac-aecf-8c37240ff158 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:17.210626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa0026-7ab3-44ac-aecf-8c37240ff158 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:17.210912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa0026-7ab3-44ac-aecf-8c37240ff158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:17.249944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:17.322706 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa0026-7ab3-44ac-aecf-8c37240ff158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:17.332600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa0026-7ab3-44ac-aecf-8c37240ff158 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:17.380683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c18f-8361-45a4-8fa9-a4d5c8fbd117 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:17.455670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8378c0-3e27-4d8e-99a8-969ffa6b47ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:17.456368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8378c0-3e27-4d8e-99a8-969ffa6b47ff None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:17.456659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8378c0-3e27-4d8e-99a8-969ffa6b47ff None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:17.456936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8378c0-3e27-4d8e-99a8-969ffa6b47ff None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:17.457672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8378c0-3e27-4d8e-99a8-969ffa6b47ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:17.459132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8378c0-3e27-4d8e-99a8-969ffa6b47ff None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:17.507127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 49/106] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 17:20:17.509643 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b448d38-be3f-4c8b-9fd3-09422d2bb3a6 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:17.509937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b448d38-be3f-4c8b-9fd3-09422d2bb3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:17.510220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b448d38-be3f-4c8b-9fd3-09422d2bb3a6 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:17.515628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 50/107] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:17.518546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 58/108] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:20:17 2022] POST /identity/v3/projects => generated 302 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:17.519474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b448d38-be3f-4c8b-9fd3-09422d2bb3a6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:17.529845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025795d3-2359-48d0-a28e-86c6e23f25c5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:17.530148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025795d3-2359-48d0-a28e-86c6e23f25c5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:17.530415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025795d3-2359-48d0-a28e-86c6e23f25c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:17.568726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-025795d3-2359-48d0-a28e-86c6e23f25c5 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:17.569940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-025795d3-2359-48d0-a28e-86c6e23f25c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:17.640709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:17.644979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:17.729402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 59/109] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:17.748967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b448d38-be3f-4c8b-9fd3-09422d2bb3a6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:17.749342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a28ef-89ae-4963-9d79-285aa77eb6e2 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:17.749642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a28ef-89ae-4963-9d79-285aa77eb6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:17.749929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a28ef-89ae-4963-9d79-285aa77eb6e2 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:17.751352 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 60/110] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:17.758096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446885e2-2cca-4cfb-8540-3141d7126ea3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:17.758385 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446885e2-2cca-4cfb-8540-3141d7126ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:17.758671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446885e2-2cca-4cfb-8540-3141d7126ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:17.791739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-446885e2-2cca-4cfb-8540-3141d7126ea3 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:17.793106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-446885e2-2cca-4cfb-8540-3141d7126ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:17.806276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb241e38-507b-40fb-97ff-82e5ea2b730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:17.807456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb241e38-507b-40fb-97ff-82e5ea2b730a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:17.807958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb241e38-507b-40fb-97ff-82e5ea2b730a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:17.808520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb241e38-507b-40fb-97ff-82e5ea2b730a None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:17.810138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb241e38-507b-40fb-97ff-82e5ea2b730a None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:17.811834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb241e38-507b-40fb-97ff-82e5ea2b730a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:17.813931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 51/111] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:17 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:17.815015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb241e38-507b-40fb-97ff-82e5ea2b730a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:17.873017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 61/112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:17.873938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446885e2-2cca-4cfb-8540-3141d7126ea3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:17.949942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb241e38-507b-40fb-97ff-82e5ea2b730a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:17.990996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:17.998943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b94d5-18da-4ddf-bf8c-0c73dbb159d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:18.000574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b94d5-18da-4ddf-bf8c-0c73dbb159d3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.001105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b94d5-18da-4ddf-bf8c-0c73dbb159d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.002748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b94d5-18da-4ddf-bf8c-0c73dbb159d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.015986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b94d5-18da-4ddf-bf8c-0c73dbb159d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:18.026827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-b12b94d5-18da-4ddf-bf8c-0c73dbb159d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:18.028384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:18.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:18.030788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:18.033229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 52/113] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:18.034029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446885e2-2cca-4cfb-8540-3141d7126ea3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:18.034774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b94d5-18da-4ddf-bf8c-0c73dbb159d3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:18.079990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ca1d1-7704-4642-b954-176c45889e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:18.080772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ca1d1-7704-4642-b954-176c45889e1f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.081004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ca1d1-7704-4642-b954-176c45889e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.081286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ca1d1-7704-4642-b954-176c45889e1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.082134 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ca1d1-7704-4642-b954-176c45889e1f None admin] RBAC: Adding query filter params (role.id=feffc80bdc3641a59c6d5d545c7fce94, scope.project.id=380cd1f0793045d280c092a275d9666a, user.id=808c0899b68e44a28407207932e719fa) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:18.082516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ca1d1-7704-4642-b954-176c45889e1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:18.084047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ca1d1-7704-4642-b954-176c45889e1f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:18.090302 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 62/114] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:17 2022] GET /identity/v3/role_assignments?user.id=808c0899b68e44a28407207932e719fa&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=feffc80bdc3641a59c6d5d545c7fce94 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:18.090861 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ca1d1-7704-4642-b954-176c45889e1f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:18.172986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b94d5-18da-4ddf-bf8c-0c73dbb159d3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:18.216105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16791216-be3e-4630-bc80-9ddb2a765e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:18.217446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791216-be3e-4630-bc80-9ddb2a765e5e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.217997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791216-be3e-4630-bc80-9ddb2a765e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.218692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791216-be3e-4630-bc80-9ddb2a765e5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.232739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16791216-be3e-4630-bc80-9ddb2a765e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-16791216-be3e-4630-bc80-9ddb2a765e5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:18.242681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:18.243565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:18.244356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:18.248184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 53/115] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:18.250149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4f577-f09e-47b7-959b-c7565a585161 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.250713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4f577-f09e-47b7-959b-c7565a585161 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.251180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4f577-f09e-47b7-959b-c7565a585161 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.252943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 54/116] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:18.258023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4f577-f09e-47b7-959b-c7565a585161 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:18.328485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ca1d1-7704-4642-b954-176c45889e1f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:18.334451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:18.373176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c0aca-0dfb-4312-b61b-6746b52be344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:18.374208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c0aca-0dfb-4312-b61b-6746b52be344 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.374689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c0aca-0dfb-4312-b61b-6746b52be344 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.375161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c0aca-0dfb-4312-b61b-6746b52be344 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.376104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c0aca-0dfb-4312-b61b-6746b52be344 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:18.377205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c0aca-0dfb-4312-b61b-6746b52be344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:18.379259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c0aca-0dfb-4312-b61b-6746b52be344 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:18.393477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 63/117] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:18.398712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fc3ed-dd18-461b-ad54-922681df58f1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.398712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fc3ed-dd18-461b-ad54-922681df58f1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.398712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fc3ed-dd18-461b-ad54-922681df58f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.411758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4f577-f09e-47b7-959b-c7565a585161 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:18.420661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-357fc3ed-dd18-461b-ad54-922681df58f1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:18.422585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-357fc3ed-dd18-461b-ad54-922681df58f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:18.447163 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:18.453986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9b942-1dca-454e-8d99-3db88247ebf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:18.454678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9b942-1dca-454e-8d99-3db88247ebf4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.455014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9b942-1dca-454e-8d99-3db88247ebf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.455297 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9b942-1dca-454e-8d99-3db88247ebf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.456107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9b942-1dca-454e-8d99-3db88247ebf4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:18.456451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9b942-1dca-454e-8d99-3db88247ebf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:18.457996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9b942-1dca-454e-8d99-3db88247ebf4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:18.467986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 55/118] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:18.469312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d02c4-482b-462a-8bd2-07cfc10bb52b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.469598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d02c4-482b-462a-8bd2-07cfc10bb52b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.469871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d02c4-482b-462a-8bd2-07cfc10bb52b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.471424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 56/119] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:18.472595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d02c4-482b-462a-8bd2-07cfc10bb52b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:18.518755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 64/120] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:18.519547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fc3ed-dd18-461b-ad54-922681df58f1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:18.615389 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d02c4-482b-462a-8bd2-07cfc10bb52b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:18.617106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:18.646478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fc3ed-dd18-461b-ad54-922681df58f1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:18.656563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09617d07-3a8a-446d-aea1-e0f13620d6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:18.657745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09617d07-3a8a-446d-aea1-e0f13620d6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.658221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09617d07-3a8a-446d-aea1-e0f13620d6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.658732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09617d07-3a8a-446d-aea1-e0f13620d6d1 None admin] PATH_INFO: `/v3/users/f2817f9598a341b1bc87577b4bc383f1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.660125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09617d07-3a8a-446d-aea1-e0f13620d6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=f2817f9598a341b1bc87577b4bc383f1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:18.661840 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09617d07-3a8a-446d-aea1-e0f13620d6d1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:18.663987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 57/121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:18 2022] GET /identity/v3/users/f2817f9598a341b1bc87577b4bc383f1 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:18.669914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f3860-7bfd-4599-92e1-7f6d655023da None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.670427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f3860-7bfd-4599-92e1-7f6d655023da None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.670898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f3860-7bfd-4599-92e1-7f6d655023da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.683455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cc929-ea74-4613-b611-584011842407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:18.685147 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cc929-ea74-4613-b611-584011842407 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.685433 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cc929-ea74-4613-b611-584011842407 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.685710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cc929-ea74-4613-b611-584011842407 None admin] PATH_INFO: `/v3/users/57de169446f4409f8d2242abb44a08b7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.686914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cc929-ea74-4613-b611-584011842407 None admin] RBAC: Authorizing `identity:get_user(user_id=57de169446f4409f8d2242abb44a08b7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:18.688398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cc929-ea74-4613-b611-584011842407 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:18.690346 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 65/122] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:18 2022] GET /identity/v3/users/57de169446f4409f8d2242abb44a08b7 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:18.691193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cc929-ea74-4613-b611-584011842407 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:18.691770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a27f3860-7bfd-4599-92e1-7f6d655023da None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:18.693274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a27f3860-7bfd-4599-92e1-7f6d655023da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:18.719704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:18.792095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 58/123] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:18.794456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f3860-7bfd-4599-92e1-7f6d655023da None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:18.804927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cc929-ea74-4613-b611-584011842407 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:18.848221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961b4f4-5924-498b-b385-7cd152aa5be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:18.849854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961b4f4-5924-498b-b385-7cd152aa5be8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.850702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961b4f4-5924-498b-b385-7cd152aa5be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.851444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961b4f4-5924-498b-b385-7cd152aa5be8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.867478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961b4f4-5924-498b-b385-7cd152aa5be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-3961b4f4-5924-498b-b385-7cd152aa5be8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:18.879069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:18.880086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:18.881052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:18.883302 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 66/124] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:18.888688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961b4f4-5924-498b-b385-7cd152aa5be8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:18.926809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:18.931123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f3860-7bfd-4599-92e1-7f6d655023da None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:18.982413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f88808-e543-4c23-a0f1-47ec41605897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:18.983775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f88808-e543-4c23-a0f1-47ec41605897 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:18.984199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f88808-e543-4c23-a0f1-47ec41605897 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:18.984680 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f88808-e543-4c23-a0f1-47ec41605897 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:18.986363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:18.987477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f88808-e543-4c23-a0f1-47ec41605897 None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:18.989517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f88808-e543-4c23-a0f1-47ec41605897 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:18.998211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 59/125] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:18 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:18.998211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f88808-e543-4c23-a0f1-47ec41605897 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:19.014931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961b4f4-5924-498b-b385-7cd152aa5be8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:19.061844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64506e-e6da-42f0-8baf-de0b330d1841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:19.063715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64506e-e6da-42f0-8baf-de0b330d1841 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:19.064214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64506e-e6da-42f0-8baf-de0b330d1841 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:19.064971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64506e-e6da-42f0-8baf-de0b330d1841 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:19.066428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64506e-e6da-42f0-8baf-de0b330d1841 None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:19.069097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64506e-e6da-42f0-8baf-de0b330d1841 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:19.076334 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 67/126] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:18 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:19.076334 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64506e-e6da-42f0-8baf-de0b330d1841 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:19.116540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f88808-e543-4c23-a0f1-47ec41605897 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:19.153509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc7d8c-df3f-469d-a62f-0f967dc5f47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:19.154548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc7d8c-df3f-469d-a62f-0f967dc5f47e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:19.154997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc7d8c-df3f-469d-a62f-0f967dc5f47e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:19.155407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc7d8c-df3f-469d-a62f-0f967dc5f47e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:19.166037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc7d8c-df3f-469d-a62f-0f967dc5f47e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-bcbc7d8c-df3f-469d-a62f-0f967dc5f47e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:19.175366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:19.176677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:19.177434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:19.179798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 60/127] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:19.180765 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc7d8c-df3f-469d-a62f-0f967dc5f47e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:19.182106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64506e-e6da-42f0-8baf-de0b330d1841 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:19.210632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:19.223916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f413e-f66e-46a0-bf56-523a0863887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:19.224840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f413e-f66e-46a0-bf56-523a0863887a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:19.225192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f413e-f66e-46a0-bf56-523a0863887a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:19.225558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f413e-f66e-46a0-bf56-523a0863887a None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:19.226446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f413e-f66e-46a0-bf56-523a0863887a None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:19.226848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f413e-f66e-46a0-bf56-523a0863887a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:19.228348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f413e-f66e-46a0-bf56-523a0863887a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:19.238905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 68/128] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:19.243643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f413e-f66e-46a0-bf56-523a0863887a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:19.263016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:19.311157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc7d8c-df3f-469d-a62f-0f967dc5f47e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:19.354689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1538ca-cef4-48ef-a838-e75bd47d196f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:19.356368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1538ca-cef4-48ef-a838-e75bd47d196f None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:19.356722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1538ca-cef4-48ef-a838-e75bd47d196f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:19.356938 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1538ca-cef4-48ef-a838-e75bd47d196f None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a/users/f2817f9598a341b1bc87577b4bc383f1/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:19.359183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1538ca-cef4-48ef-a838-e75bd47d196f None admin] RBAC: Authorizing `identity:create_grant(project_id=380cd1f0793045d280c092a275d9666a, user_id=f2817f9598a341b1bc87577b4bc383f1, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:19.360782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1538ca-cef4-48ef-a838-e75bd47d196f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:19.377445 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f413e-f66e-46a0-bf56-523a0863887a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:19.377963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 61/129] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:20:19 2022] PUT /identity/v3/projects/380cd1f0793045d280c092a275d9666a/users/f2817f9598a341b1bc87577b4bc383f1/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:19.384552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1538ca-cef4-48ef-a838-e75bd47d196f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:19.469105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c7495-df45-497d-8071-22a550462d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:19.471167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c7495-df45-497d-8071-22a550462d84 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:19.471449 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c7495-df45-497d-8071-22a550462d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:19.471723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c7495-df45-497d-8071-22a550462d84 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a/users/57de169446f4409f8d2242abb44a08b7/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:19.478737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c7495-df45-497d-8071-22a550462d84 None admin] RBAC: Authorizing `identity:create_grant(project_id=380cd1f0793045d280c092a275d9666a, user_id=57de169446f4409f8d2242abb44a08b7, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:19.480287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c7495-df45-497d-8071-22a550462d84 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:19.500948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 69/130] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:20:19 2022] PUT /identity/v3/projects/380cd1f0793045d280c092a275d9666a/users/57de169446f4409f8d2242abb44a08b7/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:19.501755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c7495-df45-497d-8071-22a550462d84 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:19.646254 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:19.653109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:19.673232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1538ca-cef4-48ef-a838-e75bd47d196f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:19.718453 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c7495-df45-497d-8071-22a550462d84 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:19.719443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b9c03-0750-494c-8dfd-26c580107463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:19.720428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b9c03-0750-494c-8dfd-26c580107463 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:19.720717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b9c03-0750-494c-8dfd-26c580107463 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:19.720998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b9c03-0750-494c-8dfd-26c580107463 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:19.721757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b9c03-0750-494c-8dfd-26c580107463 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:19.722073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b9c03-0750-494c-8dfd-26c580107463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:19.723506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b9c03-0750-494c-8dfd-26c580107463 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:19.740342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 62/131] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:19.742429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b9c03-0750-494c-8dfd-26c580107463 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:19.781075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51a647-3da7-4f6d-a573-28e6d477c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:19.781075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51a647-3da7-4f6d-a573-28e6d477c932 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:19.781310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51a647-3da7-4f6d-a573-28e6d477c932 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:19.781546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51a647-3da7-4f6d-a573-28e6d477c932 None admin] PATH_INFO: `/v3/users/f2817f9598a341b1bc87577b4bc383f1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:19.783502 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51a647-3da7-4f6d-a573-28e6d477c932 None admin] RBAC: Authorizing `identity:get_user(user_id=f2817f9598a341b1bc87577b4bc383f1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:19.785650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51a647-3da7-4f6d-a573-28e6d477c932 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:19.792033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 70/132] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:19 2022] GET /identity/v3/users/f2817f9598a341b1bc87577b4bc383f1 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:19.793331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51a647-3da7-4f6d-a573-28e6d477c932 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:19.880313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b9c03-0750-494c-8dfd-26c580107463 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:19.938824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40642b-006d-457f-8488-dbd7cf0ceab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:19.940806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40642b-006d-457f-8488-dbd7cf0ceab5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:19.941097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40642b-006d-457f-8488-dbd7cf0ceab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:19.941376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40642b-006d-457f-8488-dbd7cf0ceab5 None admin] PATH_INFO: `/v3/users/f2817f9598a341b1bc87577b4bc383f1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:19.944080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40642b-006d-457f-8488-dbd7cf0ceab5 None admin] RBAC: Authorizing `identity:get_user(user_id=f2817f9598a341b1bc87577b4bc383f1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:19.946331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40642b-006d-457f-8488-dbd7cf0ceab5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:19.951909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 63/133] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:19 2022] GET /identity/v3/users/f2817f9598a341b1bc87577b4bc383f1 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:19.963486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51a647-3da7-4f6d-a573-28e6d477c932 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:19.967795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40642b-006d-457f-8488-dbd7cf0ceab5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:20.004097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:20.066060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b84f-c1c5-49d4-af54-92bc21aa2217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:20.066864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b84f-c1c5-49d4-af54-92bc21aa2217 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:20.067156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b84f-c1c5-49d4-af54-92bc21aa2217 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:20.067440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b84f-c1c5-49d4-af54-92bc21aa2217 None admin] PATH_INFO: `/v3/projects/cba3324a947d41dc92892d0fe15c84f9` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:20.073037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3b84f-c1c5-49d4-af54-92bc21aa2217 None admin] RBAC: Authorizing `identity:get_project(project_id=cba3324a947d41dc92892d0fe15c84f9)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:20.074824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3b84f-c1c5-49d4-af54-92bc21aa2217 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:20.076624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 71/134] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:19 2022] GET /identity/v3/projects/cba3324a947d41dc92892d0fe15c84f9 => generated 312 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:20.082684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b84f-c1c5-49d4-af54-92bc21aa2217 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:20.111016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:20.164794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40642b-006d-457f-8488-dbd7cf0ceab5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:20.242503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b32d73-4719-4ff0-a9a5-f9c9ec2342e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:20.243494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b32d73-4719-4ff0-a9a5-f9c9ec2342e9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:20.243788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b32d73-4719-4ff0-a9a5-f9c9ec2342e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:20.244071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b32d73-4719-4ff0-a9a5-f9c9ec2342e9 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:20.249086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b32d73-4719-4ff0-a9a5-f9c9ec2342e9 None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:20.250522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b32d73-4719-4ff0-a9a5-f9c9ec2342e9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:20.252678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 64/135] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:19 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:20.258287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b32d73-4719-4ff0-a9a5-f9c9ec2342e9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:20.331886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b84f-c1c5-49d4-af54-92bc21aa2217 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:20.401486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:20.403204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006c076-7f9d-4afa-873b-934875c2b9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:20.404652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c076-7f9d-4afa-873b-934875c2b9e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:20.404939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c076-7f9d-4afa-873b-934875c2b9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:20.405223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c076-7f9d-4afa-873b-934875c2b9e3 None admin] PATH_INFO: `/v3/projects/cba3324a947d41dc92892d0fe15c84f9/users/f2817f9598a341b1bc87577b4bc383f1/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:20.409901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006c076-7f9d-4afa-873b-934875c2b9e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=cba3324a947d41dc92892d0fe15c84f9, user_id=f2817f9598a341b1bc87577b4bc383f1, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:20.411367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006c076-7f9d-4afa-873b-934875c2b9e3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:20.432068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b32d73-4719-4ff0-a9a5-f9c9ec2342e9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:20.437431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 72/136] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:20:20 2022] PUT /identity/v3/projects/cba3324a947d41dc92892d0fe15c84f9/users/f2817f9598a341b1bc87577b4bc383f1/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:20:20.438674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d7334-2285-4453-ba54-e1e15c7c6303 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:20.438972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d7334-2285-4453-ba54-e1e15c7c6303 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:20.439256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d7334-2285-4453-ba54-e1e15c7c6303 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:20.444245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 73/137] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:20.446566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e2e94-9712-4549-81ae-92f9287242aa None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:20.446859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e2e94-9712-4549-81ae-92f9287242aa None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:20.447142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e2e94-9712-4549-81ae-92f9287242aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:20.487828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-770e2e94-9712-4549-81ae-92f9287242aa None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:20.490177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-770e2e94-9712-4549-81ae-92f9287242aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:20.513565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f14e0c-c72c-48fa-8ae9-dfbc83cfab38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:20.515857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f14e0c-c72c-48fa-8ae9-dfbc83cfab38 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:20.516169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f14e0c-c72c-48fa-8ae9-dfbc83cfab38 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:20.516480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f14e0c-c72c-48fa-8ae9-dfbc83cfab38 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a/users/f2817f9598a341b1bc87577b4bc383f1/roles/110c20a9ff1e455d8c43ffdfbdd1447b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:20.521089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f14e0c-c72c-48fa-8ae9-dfbc83cfab38 None admin] RBAC: Authorizing `identity:create_grant(project_id=380cd1f0793045d280c092a275d9666a, user_id=f2817f9598a341b1bc87577b4bc383f1, role_id=110c20a9ff1e455d8c43ffdfbdd1447b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:20.523483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f14e0c-c72c-48fa-8ae9-dfbc83cfab38 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:20.553786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 65/138] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:20:20 2022] PUT /identity/v3/projects/380cd1f0793045d280c092a275d9666a/users/f2817f9598a341b1bc87577b4bc383f1/roles/110c20a9ff1e455d8c43ffdfbdd1447b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:20:20.608439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:20.613375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fc5d6-2259-4009-b58d-7c1f0a4922fb None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:20.613663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fc5d6-2259-4009-b58d-7c1f0a4922fb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:20.613941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fc5d6-2259-4009-b58d-7c1f0a4922fb None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:20.615813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 66/139] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:20.621931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c9894-3e60-4d38-8083-b2229bf24b43 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:20.622226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c9894-3e60-4d38-8083-b2229bf24b43 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:20.622574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c9894-3e60-4d38-8083-b2229bf24b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:20.624997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 74/140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:20.656114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-183c9894-3e60-4d38-8083-b2229bf24b43 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:20.657417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-183c9894-3e60-4d38-8083-b2229bf24b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:20.671281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e2e94-9712-4549-81ae-92f9287242aa None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:20.803652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 67/141] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:20.829307 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c9894-3e60-4d38-8083-b2229bf24b43 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:20.858062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e2e94-9712-4549-81ae-92f9287242aa None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:20.931354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef14077-5f41-425e-b7ba-7cec488f5ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:20.937139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef14077-5f41-425e-b7ba-7cec488f5ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:20.937432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef14077-5f41-425e-b7ba-7cec488f5ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:20.937716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef14077-5f41-425e-b7ba-7cec488f5ad2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:20.939349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef14077-5f41-425e-b7ba-7cec488f5ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:20.941813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef14077-5f41-425e-b7ba-7cec488f5ad2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:20.942482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 75/142] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:20.951788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef14077-5f41-425e-b7ba-7cec488f5ad2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:20.983623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:21.030743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c9894-3e60-4d38-8083-b2229bf24b43 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:21.094367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55538747-829b-40e8-812e-cd64f592f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:21.095350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55538747-829b-40e8-812e-cd64f592f177 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:21.095638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55538747-829b-40e8-812e-cd64f592f177 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:21.096256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55538747-829b-40e8-812e-cd64f592f177 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:21.116763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:21.117713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55538747-829b-40e8-812e-cd64f592f177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-55538747-829b-40e8-812e-cd64f592f177 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:21.128104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:21.129006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:21.129755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:21.130336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 68/143] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:21.138192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55538747-829b-40e8-812e-cd64f592f177 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:21.222337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef14077-5f41-425e-b7ba-7cec488f5ad2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:21.292992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55538747-829b-40e8-812e-cd64f592f177 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:21.302373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24948cb-bf5a-4bf2-aed9-02fb670c7159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:21.303248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24948cb-bf5a-4bf2-aed9-02fb670c7159 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:21.303677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24948cb-bf5a-4bf2-aed9-02fb670c7159 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:21.304094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24948cb-bf5a-4bf2-aed9-02fb670c7159 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:21.304985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24948cb-bf5a-4bf2-aed9-02fb670c7159 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:21.306608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24948cb-bf5a-4bf2-aed9-02fb670c7159 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:21.353555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdd5c3-5c76-4c21-80d1-8ddcb6235cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:21.354234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdd5c3-5c76-4c21-80d1-8ddcb6235cba None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:21.354519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdd5c3-5c76-4c21-80d1-8ddcb6235cba None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:21.354796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdd5c3-5c76-4c21-80d1-8ddcb6235cba None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:21.355768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdd5c3-5c76-4c21-80d1-8ddcb6235cba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:21.356016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 76/144] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:20:20 2022] POST /identity/v3/users => generated 283 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:21.356123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdd5c3-5c76-4c21-80d1-8ddcb6235cba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:21.357601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdd5c3-5c76-4c21-80d1-8ddcb6235cba None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:21.365616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:21.379093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 69/145] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:21.384512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24948cb-bf5a-4bf2-aed9-02fb670c7159 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:21.528041 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24948cb-bf5a-4bf2-aed9-02fb670c7159 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:21.553090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:21.566525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3e13c-24b3-475d-813e-102366dc3110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:21.567627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e13c-24b3-475d-813e-102366dc3110 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:21.568072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e13c-24b3-475d-813e-102366dc3110 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:21.568530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e13c-24b3-475d-813e-102366dc3110 None admin] PATH_INFO: `/v3/users/808c0899b68e44a28407207932e719fa` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:21.569883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3e13c-24b3-475d-813e-102366dc3110 None admin] RBAC: Authorizing `identity:get_user(user_id=808c0899b68e44a28407207932e719fa)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:21.571484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3e13c-24b3-475d-813e-102366dc3110 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:21.573664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 77/146] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:21 2022] GET /identity/v3/users/808c0899b68e44a28407207932e719fa => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:21.579965 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdd5c3-5c76-4c21-80d1-8ddcb6235cba None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:21.716806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdd5c3-5c76-4c21-80d1-8ddcb6235cba None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:21.765036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e7efe-aee8-48a5-8229-5372e35201ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:21.765935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7efe-aee8-48a5-8229-5372e35201ae None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:21.766224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7efe-aee8-48a5-8229-5372e35201ae None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:21.766511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7efe-aee8-48a5-8229-5372e35201ae None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:21.767790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e7efe-aee8-48a5-8229-5372e35201ae None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:21.769271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e7efe-aee8-48a5-8229-5372e35201ae None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:21.770990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 70/147] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:21 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:21.776976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3e13c-24b3-475d-813e-102366dc3110 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:21.911588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3e13c-24b3-475d-813e-102366dc3110 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:21.963312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160af1e-5d0d-4a9c-83c7-4560eb1c7cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:21.964203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160af1e-5d0d-4a9c-83c7-4560eb1c7cb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:21.964518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160af1e-5d0d-4a9c-83c7-4560eb1c7cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:21.965509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160af1e-5d0d-4a9c-83c7-4560eb1c7cb6 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a/users/808c0899b68e44a28407207932e719fa/roles/feffc80bdc3641a59c6d5d545c7fce94` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:21.969423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160af1e-5d0d-4a9c-83c7-4560eb1c7cb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=380cd1f0793045d280c092a275d9666a, user_id=808c0899b68e44a28407207932e719fa, role_id=feffc80bdc3641a59c6d5d545c7fce94)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:21.970881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160af1e-5d0d-4a9c-83c7-4560eb1c7cb6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:21.988827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 78/148] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:20:21 2022] PUT /identity/v3/projects/380cd1f0793045d280c092a275d9666a/users/808c0899b68e44a28407207932e719fa/roles/feffc80bdc3641a59c6d5d545c7fce94 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:22.328999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92140a-7d60-481a-be32-513b2217dca0 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:22.329501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92140a-7d60-481a-be32-513b2217dca0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:22.329501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92140a-7d60-481a-be32-513b2217dca0 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:22.330849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 71/149] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:22.336329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d919479-9336-4595-a402-da9a0a26a02b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:22.336657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d919479-9336-4595-a402-da9a0a26a02b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:22.336936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d919479-9336-4595-a402-da9a0a26a02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:22.361430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7d919479-9336-4595-a402-da9a0a26a02b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:22.362643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7d919479-9336-4595-a402-da9a0a26a02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:22.417452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:22.446719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 79/150] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:22.485707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92140a-7d60-481a-be32-513b2217dca0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:22.581119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:22.597061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edaa70-4bb0-4a73-b803-eeb319964822 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:22.598149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edaa70-4bb0-4a73-b803-eeb319964822 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:22.598627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edaa70-4bb0-4a73-b803-eeb319964822 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:22.603235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 80/151] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:22.608784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8d08d-eca6-4309-89e3-45fdfb880fe4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:22.609297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8d08d-eca6-4309-89e3-45fdfb880fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:22.609954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8d08d-eca6-4309-89e3-45fdfb880fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:22.630132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f1c8d08d-eca6-4309-89e3-45fdfb880fe4 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:22.631636 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f1c8d08d-eca6-4309-89e3-45fdfb880fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:22.634534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92140a-7d60-481a-be32-513b2217dca0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:22.679325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee6e2e-005f-474f-b121-82c3ad3395a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:22.680137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee6e2e-005f-474f-b121-82c3ad3395a2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:22.680445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee6e2e-005f-474f-b121-82c3ad3395a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:22.680726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee6e2e-005f-474f-b121-82c3ad3395a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:22.704412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee6e2e-005f-474f-b121-82c3ad3395a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-10ee6e2e-005f-474f-b121-82c3ad3395a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:22.713742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:22.715797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:22.716611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 72/152] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:22.728915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee6e2e-005f-474f-b121-82c3ad3395a2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:22.761259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 81/153] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:22.805057 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8d08d-eca6-4309-89e3-45fdfb880fe4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:22.956500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee6e2e-005f-474f-b121-82c3ad3395a2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:23.011519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:23.025119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8d08d-eca6-4309-89e3-45fdfb880fe4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:23.029978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf8637-2fb6-4a0b-9b51-fe7e6eeaad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:23.030973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf8637-2fb6-4a0b-9b51-fe7e6eeaad7e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:23.031464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf8637-2fb6-4a0b-9b51-fe7e6eeaad7e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:23.031935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf8637-2fb6-4a0b-9b51-fe7e6eeaad7e None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:23.032915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf8637-2fb6-4a0b-9b51-fe7e6eeaad7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:23.033421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf8637-2fb6-4a0b-9b51-fe7e6eeaad7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:23.035037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf8637-2fb6-4a0b-9b51-fe7e6eeaad7e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:23.041939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:23.055772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 73/154] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:23.062832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf8637-2fb6-4a0b-9b51-fe7e6eeaad7e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:23.092032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916c752-4439-47b4-ab5e-31bcffc31751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:23.093237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c752-4439-47b4-ab5e-31bcffc31751 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:23.093717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c752-4439-47b4-ab5e-31bcffc31751 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:23.094184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c752-4439-47b4-ab5e-31bcffc31751 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:23.124945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916c752-4439-47b4-ab5e-31bcffc31751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6916c752-4439-47b4-ab5e-31bcffc31751 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:23.133828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:23.134673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:23.135588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:23.148016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 82/155] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:23.154549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916c752-4439-47b4-ab5e-31bcffc31751 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:23.298387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf8637-2fb6-4a0b-9b51-fe7e6eeaad7e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:23.385050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916c752-4439-47b4-ab5e-31bcffc31751 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:23.405603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540ac18-bea3-4b95-a47b-255c799741e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:23.407189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540ac18-bea3-4b95-a47b-255c799741e5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:23.407672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540ac18-bea3-4b95-a47b-255c799741e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:23.408156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540ac18-bea3-4b95-a47b-255c799741e5 None admin] PATH_INFO: `/v3/users/f2817f9598a341b1bc87577b4bc383f1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:23.409632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540ac18-bea3-4b95-a47b-255c799741e5 None admin] RBAC: Authorizing `identity:get_user(user_id=f2817f9598a341b1bc87577b4bc383f1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:23.411318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540ac18-bea3-4b95-a47b-255c799741e5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:23.413222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 74/156] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:23 2022] GET /identity/v3/users/f2817f9598a341b1bc87577b4bc383f1 => generated 275 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:23.425354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540ac18-bea3-4b95-a47b-255c799741e5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:23.448459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a84647-645c-4a39-9211-c09b96dc8b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:23.449214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a84647-645c-4a39-9211-c09b96dc8b3a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:23.450104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a84647-645c-4a39-9211-c09b96dc8b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:23.450104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a84647-645c-4a39-9211-c09b96dc8b3a None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:23.451003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a84647-645c-4a39-9211-c09b96dc8b3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:23.451327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a84647-645c-4a39-9211-c09b96dc8b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:23.452805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a84647-645c-4a39-9211-c09b96dc8b3a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:23.469428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 83/157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:23.475162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a84647-645c-4a39-9211-c09b96dc8b3a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:23.537658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:23.572738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:23.671136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a84647-645c-4a39-9211-c09b96dc8b3a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:23.679917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540ac18-bea3-4b95-a47b-255c799741e5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:23.734030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a0ca4-7d5f-47a5-b7f9-49843331d73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:23.734924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a0ca4-7d5f-47a5-b7f9-49843331d73c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:23.735214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a0ca4-7d5f-47a5-b7f9-49843331d73c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:23.735470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a0ca4-7d5f-47a5-b7f9-49843331d73c None admin] PATH_INFO: `/v3/users/57de169446f4409f8d2242abb44a08b7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:23.739988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a0ca4-7d5f-47a5-b7f9-49843331d73c None admin] RBAC: Authorizing `identity:get_user(user_id=57de169446f4409f8d2242abb44a08b7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:23.741520 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a0ca4-7d5f-47a5-b7f9-49843331d73c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:23.743363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 84/158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:23 2022] GET /identity/v3/users/57de169446f4409f8d2242abb44a08b7 => generated 247 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:23.749513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a0ca4-7d5f-47a5-b7f9-49843331d73c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:23.752214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed35da-c26d-4960-bb97-3472b133f85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:23.753144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed35da-c26d-4960-bb97-3472b133f85b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:23.753624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed35da-c26d-4960-bb97-3472b133f85b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:23.754131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed35da-c26d-4960-bb97-3472b133f85b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:23.905185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:23.909596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed35da-c26d-4960-bb97-3472b133f85b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:23.911091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed35da-c26d-4960-bb97-3472b133f85b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:23.911272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a0ca4-7d5f-47a5-b7f9-49843331d73c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:23.981653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5163682-9fe4-4b60-90da-81464bf97933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:23.982076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5163682-9fe4-4b60-90da-81464bf97933 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:23.982356 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5163682-9fe4-4b60-90da-81464bf97933 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:23.982629 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5163682-9fe4-4b60-90da-81464bf97933 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:24.020686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:24.056921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5163682-9fe4-4b60-90da-81464bf97933 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:24.059304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5163682-9fe4-4b60-90da-81464bf97933 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:24.076593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 75/159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:24.082007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed35da-c26d-4960-bb97-3472b133f85b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:24.166103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:24.171805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 85/160] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:24.177206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5163682-9fe4-4b60-90da-81464bf97933 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:24.228026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed35da-c26d-4960-bb97-3472b133f85b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:24.282195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5163682-9fe4-4b60-90da-81464bf97933 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:24.287419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be64054-b80d-49d5-8796-67642d7fc282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:24.288324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be64054-b80d-49d5-8796-67642d7fc282 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:24.288618 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be64054-b80d-49d5-8796-67642d7fc282 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:24.288895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be64054-b80d-49d5-8796-67642d7fc282 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:24.293081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be64054-b80d-49d5-8796-67642d7fc282 None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:24.294543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be64054-b80d-49d5-8796-67642d7fc282 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:24.296647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 76/161] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:24 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:24.302051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be64054-b80d-49d5-8796-67642d7fc282 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:24.318836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffb5fb-b510-4d06-8d07-bc886531aaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:24.319696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb5fb-b510-4d06-8d07-bc886531aaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:24.319977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb5fb-b510-4d06-8d07-bc886531aaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:24.320245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb5fb-b510-4d06-8d07-bc886531aaf3 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:24.321507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffb5fb-b510-4d06-8d07-bc886531aaf3 None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:24.322966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffb5fb-b510-4d06-8d07-bc886531aaf3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:24.326089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 86/162] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:24 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:24.332049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffb5fb-b510-4d06-8d07-bc886531aaf3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:24.351188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:24.489717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:24.520277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be64054-b80d-49d5-8796-67642d7fc282 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:24.578467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffb5fb-b510-4d06-8d07-bc886531aaf3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:24.609549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa6373-360c-4a28-b0ac-da02e17391ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:24.610456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6373-360c-4a28-b0ac-da02e17391ec None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:24.610880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6373-360c-4a28-b0ac-da02e17391ec None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:24.611295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6373-360c-4a28-b0ac-da02e17391ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:24.612322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa6373-360c-4a28-b0ac-da02e17391ec None admin] RBAC: Adding query filter params (role.id=d2c2845de1d548bb9cfb46f70619b3f8, scope.project.id=380cd1f0793045d280c092a275d9666a, user.id=f2817f9598a341b1bc87577b4bc383f1) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:24.612860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa6373-360c-4a28-b0ac-da02e17391ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:24.614490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa6373-360c-4a28-b0ac-da02e17391ec None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:24.632555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 77/163] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:24 2022] GET /identity/v3/role_assignments?user.id=f2817f9598a341b1bc87577b4bc383f1&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:24.633872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12740d-4489-4b53-aece-b34a6234bdb1 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:24.634273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12740d-4489-4b53-aece-b34a6234bdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:24.634550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12740d-4489-4b53-aece-b34a6234bdb1 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:24.635939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 78/164] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:24.637835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860acda-7da3-46f2-a7fd-0e85fddcfdbe None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:24.638114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860acda-7da3-46f2-a7fd-0e85fddcfdbe None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:24.638388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860acda-7da3-46f2-a7fd-0e85fddcfdbe None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:24.639629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 79/165] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:24.650129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349ec57-df3d-40b4-8463-3970c0dc28ec None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:24.650415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349ec57-df3d-40b4-8463-3970c0dc28ec None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:24.650867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349ec57-df3d-40b4-8463-3970c0dc28ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:24.677035 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af0157-9547-4551-b95f-811aa5594075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:24.677714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0157-9547-4551-b95f-811aa5594075 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:24.677999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0157-9547-4551-b95f-811aa5594075 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:24.678213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7349ec57-df3d-40b4-8463-3970c0dc28ec None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:24.678327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0157-9547-4551-b95f-811aa5594075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:24.679120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af0157-9547-4551-b95f-811aa5594075 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=380cd1f0793045d280c092a275d9666a, user.id=57de169446f4409f8d2242abb44a08b7) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:24.679435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af0157-9547-4551-b95f-811aa5594075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:24.679966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7349ec57-df3d-40b4-8463-3970c0dc28ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:24.680973 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af0157-9547-4551-b95f-811aa5594075 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:24.691788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 87/166] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:24 2022] GET /identity/v3/role_assignments?user.id=57de169446f4409f8d2242abb44a08b7&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 608 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:24.693495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063a96f-3708-4dd6-a68e-4237385a7025 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:24.693952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063a96f-3708-4dd6-a68e-4237385a7025 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:24.694381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063a96f-3708-4dd6-a68e-4237385a7025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:24.725412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4063a96f-3708-4dd6-a68e-4237385a7025 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:24.726881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4063a96f-3708-4dd6-a68e-4237385a7025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:24.862461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 88/167] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:24.866551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 80/168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:24.891297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063a96f-3708-4dd6-a68e-4237385a7025 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:24.936981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349ec57-df3d-40b4-8463-3970c0dc28ec None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:24.985239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:25.007157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:25.080879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063a96f-3708-4dd6-a68e-4237385a7025 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:25.146196 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775665c3-d905-4c58-b989-7e9e85d4fa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:25.147018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775665c3-d905-4c58-b989-7e9e85d4fa2f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:25.147306 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775665c3-d905-4c58-b989-7e9e85d4fa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:25.147587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775665c3-d905-4c58-b989-7e9e85d4fa2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:25.170184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349ec57-df3d-40b4-8463-3970c0dc28ec None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:25.171763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775665c3-d905-4c58-b989-7e9e85d4fa2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-775665c3-d905-4c58-b989-7e9e85d4fa2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:25.187222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:25.188435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:25.189811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 89/169] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:25.194866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775665c3-d905-4c58-b989-7e9e85d4fa2f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:25.239491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c8341-005f-4155-b3c3-8422ec5c1632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:25.240340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c8341-005f-4155-b3c3-8422ec5c1632 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:25.240616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c8341-005f-4155-b3c3-8422ec5c1632 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:25.240891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c8341-005f-4155-b3c3-8422ec5c1632 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:25.259943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c8341-005f-4155-b3c3-8422ec5c1632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-307c8341-005f-4155-b3c3-8422ec5c1632 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:25.278597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:25.279531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:25.280393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 81/170] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:25.300750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c8341-005f-4155-b3c3-8422ec5c1632 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:25.387860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775665c3-d905-4c58-b989-7e9e85d4fa2f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:25.429289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:25.451977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:25.460230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c8341-005f-4155-b3c3-8422ec5c1632 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:25.487718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d73b0f-ea37-40c1-aa4c-6bd3459be9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:25.488432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d73b0f-ea37-40c1-aa4c-6bd3459be9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:25.488719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d73b0f-ea37-40c1-aa4c-6bd3459be9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:25.488999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d73b0f-ea37-40c1-aa4c-6bd3459be9b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:25.489765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d73b0f-ea37-40c1-aa4c-6bd3459be9b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:25.490075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d73b0f-ea37-40c1-aa4c-6bd3459be9b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:25.491466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d73b0f-ea37-40c1-aa4c-6bd3459be9b9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:25.504653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 90/171] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:25.506488 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d530f-e728-4a12-9aa4-e607891e096a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:25.507180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d530f-e728-4a12-9aa4-e607891e096a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:25.507465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d530f-e728-4a12-9aa4-e607891e096a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:25.507744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d530f-e728-4a12-9aa4-e607891e096a None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:25.512032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d530f-e728-4a12-9aa4-e607891e096a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:25.512032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d530f-e728-4a12-9aa4-e607891e096a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:25.512032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d530f-e728-4a12-9aa4-e607891e096a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:25.513266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d73b0f-ea37-40c1-aa4c-6bd3459be9b9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:25.532587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 82/172] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:25.537936 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d530f-e728-4a12-9aa4-e607891e096a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:25.684447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d73b0f-ea37-40c1-aa4c-6bd3459be9b9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:25.719909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75322066-cf5c-4319-b65b-c687b3f7a4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:25.721097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75322066-cf5c-4319-b65b-c687b3f7a4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:25.721524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75322066-cf5c-4319-b65b-c687b3f7a4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:25.721941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75322066-cf5c-4319-b65b-c687b3f7a4c4 None admin] PATH_INFO: `/v3/users/f2817f9598a341b1bc87577b4bc383f1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:25.723291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75322066-cf5c-4319-b65b-c687b3f7a4c4 None admin] RBAC: Authorizing `identity:get_user(user_id=f2817f9598a341b1bc87577b4bc383f1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:25.724927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75322066-cf5c-4319-b65b-c687b3f7a4c4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:25.727344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 91/173] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:25 2022] GET /identity/v3/users/f2817f9598a341b1bc87577b4bc383f1 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:25.728807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb6eef-157b-43b0-9982-4df1b9badca5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:25.729253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb6eef-157b-43b0-9982-4df1b9badca5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:25.729750 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb6eef-157b-43b0-9982-4df1b9badca5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:25.731832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 92/174] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:25.735493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d530f-e728-4a12-9aa4-e607891e096a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:25.741682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb6eef-157b-43b0-9982-4df1b9badca5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:25.792334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a575fa-14b4-4104-972b-8f5a2b03a83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:25.794267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a575fa-14b4-4104-972b-8f5a2b03a83a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:25.794683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a575fa-14b4-4104-972b-8f5a2b03a83a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:25.795116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a575fa-14b4-4104-972b-8f5a2b03a83a None admin] PATH_INFO: `/v3/users/f2817f9598a341b1bc87577b4bc383f1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:25.800254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a575fa-14b4-4104-972b-8f5a2b03a83a None admin] RBAC: Authorizing `identity:get_user(user_id=f2817f9598a341b1bc87577b4bc383f1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:25.801816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a575fa-14b4-4104-972b-8f5a2b03a83a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:25.803754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 83/175] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:25 2022] GET /identity/v3/users/f2817f9598a341b1bc87577b4bc383f1 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:25.805380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ef90f-53db-4fc4-96f3-f952f975c91b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:25.805807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ef90f-53db-4fc4-96f3-f952f975c91b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:25.806225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ef90f-53db-4fc4-96f3-f952f975c91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:25.858200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3f2ef90f-53db-4fc4-96f3-f952f975c91b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:25.859422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3f2ef90f-53db-4fc4-96f3-f952f975c91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:25.861630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:25.892405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:25.958684 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 84/176] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:25.959466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ef90f-53db-4fc4-96f3-f952f975c91b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:26.017715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb6eef-157b-43b0-9982-4df1b9badca5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:26.101910 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916bee4-d60d-4731-b52a-1eaea8f94321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:26.103120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916bee4-d60d-4731-b52a-1eaea8f94321 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.103894 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916bee4-d60d-4731-b52a-1eaea8f94321 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.104660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916bee4-d60d-4731-b52a-1eaea8f94321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.109645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ef90f-53db-4fc4-96f3-f952f975c91b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:26.146365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb21993-045e-41c4-8370-e7d79e76f5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:26.147562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21993-045e-41c4-8370-e7d79e76f5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.148902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21993-045e-41c4-8370-e7d79e76f5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.149392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21993-045e-41c4-8370-e7d79e76f5b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.160188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:26.193659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916bee4-d60d-4731-b52a-1eaea8f94321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:26.196209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916bee4-d60d-4731-b52a-1eaea8f94321 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:26.231047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb21993-045e-41c4-8370-e7d79e76f5b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:26.232963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb21993-045e-41c4-8370-e7d79e76f5b3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:26.265697 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:26.319499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 93/177] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:26.320609 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916bee4-d60d-4731-b52a-1eaea8f94321 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:26.355371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 85/178] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:26.356695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96e9c-4dc3-404a-b6f8-4967875b5af8 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.356980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96e9c-4dc3-404a-b6f8-4967875b5af8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.357261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96e9c-4dc3-404a-b6f8-4967875b5af8 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.358533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 86/179] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:26.359779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567dcd8-6804-4c30-96ee-e295795514d1 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.359883 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567dcd8-6804-4c30-96ee-e295795514d1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.360121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567dcd8-6804-4c30-96ee-e295795514d1 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.364341 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 87/180] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:26.364341 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567dcd8-6804-4c30-96ee-e295795514d1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:26.404260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:26.424054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916bee4-d60d-4731-b52a-1eaea8f94321 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:26.458682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f64c0-1ebd-4556-9608-e76ef7c1439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:26.459476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f64c0-1ebd-4556-9608-e76ef7c1439d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.459756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f64c0-1ebd-4556-9608-e76ef7c1439d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.460031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f64c0-1ebd-4556-9608-e76ef7c1439d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.466525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567dcd8-6804-4c30-96ee-e295795514d1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:26.470839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f64c0-1ebd-4556-9608-e76ef7c1439d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-759f64c0-1ebd-4556-9608-e76ef7c1439d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:26.478616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:26.479493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:26.480798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 94/181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:26.482872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b7587-db45-4d45-a316-97cac85a593f None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.483107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b7587-db45-4d45-a316-97cac85a593f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.483388 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b7587-db45-4d45-a316-97cac85a593f None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.484748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 95/182] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:26.485447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b7587-db45-4d45-a316-97cac85a593f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:26.501425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:26.501977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bb598-4b76-467d-b6bb-66b326303854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:26.503021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bb598-4b76-467d-b6bb-66b326303854 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.503386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bb598-4b76-467d-b6bb-66b326303854 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.503840 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bb598-4b76-467d-b6bb-66b326303854 None admin] PATH_INFO: `/v3/projects/cba3324a947d41dc92892d0fe15c84f9` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.505063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bb598-4b76-467d-b6bb-66b326303854 None admin] RBAC: Authorizing `identity:get_project(project_id=cba3324a947d41dc92892d0fe15c84f9)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:26.506487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bb598-4b76-467d-b6bb-66b326303854 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:26.508162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 88/183] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:26 2022] GET /identity/v3/projects/cba3324a947d41dc92892d0fe15c84f9 => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:26.509373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45813dce-ded4-400a-8add-87098011c4b2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.509653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45813dce-ded4-400a-8add-87098011c4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.509926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45813dce-ded4-400a-8add-87098011c4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.526171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-45813dce-ded4-400a-8add-87098011c4b2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:26.527358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-45813dce-ded4-400a-8add-87098011c4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:26.589103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b7587-db45-4d45-a316-97cac85a593f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:26.594042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 89/184] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:26.595302 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c2068-4a54-4c05-b9d8-4f3d6930ea10 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.595613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c2068-4a54-4c05-b9d8-4f3d6930ea10 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.595873 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c2068-4a54-4c05-b9d8-4f3d6930ea10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.611984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-181c2068-4a54-4c05-b9d8-4f3d6930ea10 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:26.613193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-181c2068-4a54-4c05-b9d8-4f3d6930ea10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:26.632678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7b9d1-7c3d-49dc-9f12-7692d210ece2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:26.633534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7b9d1-7c3d-49dc-9f12-7692d210ece2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.633810 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7b9d1-7c3d-49dc-9f12-7692d210ece2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.634079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7b9d1-7c3d-49dc-9f12-7692d210ece2 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.635233 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7b9d1-7c3d-49dc-9f12-7692d210ece2 None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:26.637410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7b9d1-7c3d-49dc-9f12-7692d210ece2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:26.639092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 96/185] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:26 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:26.639840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7b9d1-7c3d-49dc-9f12-7692d210ece2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:26.647724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:26.686120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 90/186] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:26.693800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d010c99-8953-4153-a509-bfb5bc5c9ae5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.693800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d010c99-8953-4153-a509-bfb5bc5c9ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.693800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d010c99-8953-4153-a509-bfb5bc5c9ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.704404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5d010c99-8953-4153-a509-bfb5bc5c9ae5 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:26.706065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5d010c99-8953-4153-a509-bfb5bc5c9ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:26.731937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:26.751843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7b9d1-7c3d-49dc-9f12-7692d210ece2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:26.776779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 91/187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:26.787221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d010c99-8953-4153-a509-bfb5bc5c9ae5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:26.791527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16090f-eef5-49d0-a985-f06c6cc7ca75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:26.792326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16090f-eef5-49d0-a985-f06c6cc7ca75 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.792506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16090f-eef5-49d0-a985-f06c6cc7ca75 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.792803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16090f-eef5-49d0-a985-f06c6cc7ca75 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.793610 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16090f-eef5-49d0-a985-f06c6cc7ca75 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:26.793847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16090f-eef5-49d0-a985-f06c6cc7ca75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:26.795228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16090f-eef5-49d0-a985-f06c6cc7ca75 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:26.803620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 97/188] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:26.804466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16090f-eef5-49d0-a985-f06c6cc7ca75 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:26.870029 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:26.877017 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d010c99-8953-4153-a509-bfb5bc5c9ae5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:26.903670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16090f-eef5-49d0-a985-f06c6cc7ca75 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:26.911232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870f27d-fb91-4f6d-bd30-d1975b91368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:26.911887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870f27d-fb91-4f6d-bd30-d1975b91368c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.912173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870f27d-fb91-4f6d-bd30-d1975b91368c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.912485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870f27d-fb91-4f6d-bd30-d1975b91368c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.913330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870f27d-fb91-4f6d-bd30-d1975b91368c None admin] RBAC: Adding query filter params (role.id=d2c2845de1d548bb9cfb46f70619b3f8, scope.project.id=cba3324a947d41dc92892d0fe15c84f9, user.id=f2817f9598a341b1bc87577b4bc383f1) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:26.913645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870f27d-fb91-4f6d-bd30-d1975b91368c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:26.915093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870f27d-fb91-4f6d-bd30-d1975b91368c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:26.921759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 92/189] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:26 2022] GET /identity/v3/role_assignments?user.id=f2817f9598a341b1bc87577b4bc383f1&scope.project.id=cba3324a947d41dc92892d0fe15c84f9&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:26.922629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870f27d-fb91-4f6d-bd30-d1975b91368c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:26.945903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57705a44-6f1b-47d0-90a6-31dd69bafa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:26.946723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57705a44-6f1b-47d0-90a6-31dd69bafa0f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:26.947005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57705a44-6f1b-47d0-90a6-31dd69bafa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:26.947281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57705a44-6f1b-47d0-90a6-31dd69bafa0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:26.961029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57705a44-6f1b-47d0-90a6-31dd69bafa0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-57705a44-6f1b-47d0-90a6-31dd69bafa0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:26.971222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:26.972214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:26.973138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:26.973738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 98/190] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:26.974355 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57705a44-6f1b-47d0-90a6-31dd69bafa0f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:26.978080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:27.044252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870f27d-fb91-4f6d-bd30-d1975b91368c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:27.088899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5be3b1-117f-41d3-a67a-a8b81e1436f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:27.088899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5be3b1-117f-41d3-a67a-a8b81e1436f2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:27.088899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5be3b1-117f-41d3-a67a-a8b81e1436f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:27.088899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5be3b1-117f-41d3-a67a-a8b81e1436f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:27.089333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5be3b1-117f-41d3-a67a-a8b81e1436f2 None admin] RBAC: Adding query filter params (role.id=110c20a9ff1e455d8c43ffdfbdd1447b, scope.project.id=380cd1f0793045d280c092a275d9666a, user.id=f2817f9598a341b1bc87577b4bc383f1) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:27.089333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5be3b1-117f-41d3-a67a-a8b81e1436f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:27.091339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5be3b1-117f-41d3-a67a-a8b81e1436f2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:27.098318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 93/191] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:26 2022] GET /identity/v3/role_assignments?user.id=f2817f9598a341b1bc87577b4bc383f1&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=110c20a9ff1e455d8c43ffdfbdd1447b => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:27.100483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5be3b1-117f-41d3-a67a-a8b81e1436f2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:27.101190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57705a44-6f1b-47d0-90a6-31dd69bafa0f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:27.135679 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:27.157184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039a232-3ead-44fd-8da4-3c95de16ecba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:27.158026 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039a232-3ead-44fd-8da4-3c95de16ecba None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:27.158362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039a232-3ead-44fd-8da4-3c95de16ecba None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:27.158639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039a232-3ead-44fd-8da4-3c95de16ecba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:27.173177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039a232-3ead-44fd-8da4-3c95de16ecba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b039a232-3ead-44fd-8da4-3c95de16ecba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:27.181482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:27.182427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:27.183856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 99/192] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:27.184550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039a232-3ead-44fd-8da4-3c95de16ecba None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:27.234373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5be3b1-117f-41d3-a67a-a8b81e1436f2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:27.265057 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:27.275072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a17c72-673f-48e9-8138-74fe0e9e4d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:27.275616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a17c72-673f-48e9-8138-74fe0e9e4d6b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:27.276075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a17c72-673f-48e9-8138-74fe0e9e4d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:27.276185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a17c72-673f-48e9-8138-74fe0e9e4d6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:27.278490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a17c72-673f-48e9-8138-74fe0e9e4d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:27.279481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a17c72-673f-48e9-8138-74fe0e9e4d6b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:27.281155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 94/193] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:27.286524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a17c72-673f-48e9-8138-74fe0e9e4d6b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:27.323929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039a232-3ead-44fd-8da4-3c95de16ecba None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:27.362827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bae86-3240-4d70-93b4-ba9ff73190c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:27.363717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bae86-3240-4d70-93b4-ba9ff73190c0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:27.364004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bae86-3240-4d70-93b4-ba9ff73190c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:27.364316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bae86-3240-4d70-93b4-ba9ff73190c0 None admin] PATH_INFO: `/v3/users/808c0899b68e44a28407207932e719fa` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:27.365517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bae86-3240-4d70-93b4-ba9ff73190c0 None admin] RBAC: Authorizing `identity:get_user(user_id=808c0899b68e44a28407207932e719fa)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:27.366988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bae86-3240-4d70-93b4-ba9ff73190c0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:27.368659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 100/194] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:27 2022] GET /identity/v3/users/808c0899b68e44a28407207932e719fa => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:27.369425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bae86-3240-4d70-93b4-ba9ff73190c0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:27.385307 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:27.395016 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a17c72-673f-48e9-8138-74fe0e9e4d6b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:27.429855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5e725-6ba3-42ee-8908-fe060148df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:27.430516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5e725-6ba3-42ee-8908-fe060148df33 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:27.430797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5e725-6ba3-42ee-8908-fe060148df33 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:27.431068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5e725-6ba3-42ee-8908-fe060148df33 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:27.431812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5e725-6ba3-42ee-8908-fe060148df33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:27.432116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5e725-6ba3-42ee-8908-fe060148df33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:27.433571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5e725-6ba3-42ee-8908-fe060148df33 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:27.441792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 95/195] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:27.442558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5e725-6ba3-42ee-8908-fe060148df33 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:27.472255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bae86-3240-4d70-93b4-ba9ff73190c0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:27.500446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:27.506198 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023ea9e-5abf-491d-9efe-1df6cdb8c7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:27.506892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023ea9e-5abf-491d-9efe-1df6cdb8c7be None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:27.507176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023ea9e-5abf-491d-9efe-1df6cdb8c7be None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:27.507453 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023ea9e-5abf-491d-9efe-1df6cdb8c7be None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:27.508201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023ea9e-5abf-491d-9efe-1df6cdb8c7be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:27.508529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023ea9e-5abf-491d-9efe-1df6cdb8c7be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:27.509922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023ea9e-5abf-491d-9efe-1df6cdb8c7be None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:27.518024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 101/196] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:27.518806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023ea9e-5abf-491d-9efe-1df6cdb8c7be None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:27.543417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5e725-6ba3-42ee-8908-fe060148df33 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:27.576834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1a246-d1ae-435c-ade6-f9ccc5f99013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:27.577440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1a246-d1ae-435c-ade6-f9ccc5f99013 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:27.577733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1a246-d1ae-435c-ade6-f9ccc5f99013 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:27.578013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1a246-d1ae-435c-ade6-f9ccc5f99013 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:27.578726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1a246-d1ae-435c-ade6-f9ccc5f99013 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:27.580166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1a246-d1ae-435c-ade6-f9ccc5f99013 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:27.601122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 96/197] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:20:27 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:27.601981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1a246-d1ae-435c-ade6-f9ccc5f99013 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:27.618200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:27.627045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023ea9e-5abf-491d-9efe-1df6cdb8c7be None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:27.671934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab75ed-1bbb-4f10-82de-7cbe19137e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:27.672638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab75ed-1bbb-4f10-82de-7cbe19137e31 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:27.672929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab75ed-1bbb-4f10-82de-7cbe19137e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:27.673210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab75ed-1bbb-4f10-82de-7cbe19137e31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:27.718491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1a246-d1ae-435c-ade6-f9ccc5f99013 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:27.740644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:27.752814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab75ed-1bbb-4f10-82de-7cbe19137e31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:27.754308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab75ed-1bbb-4f10-82de-7cbe19137e31 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:27.762047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8864439-87cc-4632-a872-510961d1201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:27.763185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8864439-87cc-4632-a872-510961d1201a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:27.763393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8864439-87cc-4632-a872-510961d1201a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:27.764843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8864439-87cc-4632-a872-510961d1201a None admin] PATH_INFO: `/v3/users/d04590925e4941678119372638c4b9e3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:27.781464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8864439-87cc-4632-a872-510961d1201a None admin] RBAC: Authorizing `identity:get_user(user_id=d04590925e4941678119372638c4b9e3)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:27.783017 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8864439-87cc-4632-a872-510961d1201a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:27.784887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 97/198] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:27 2022] GET /identity/v3/users/d04590925e4941678119372638c4b9e3 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:27.785661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8864439-87cc-4632-a872-510961d1201a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:27.869509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:27.887178 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 102/199] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:27.888039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab75ed-1bbb-4f10-82de-7cbe19137e31 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:27.915350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8864439-87cc-4632-a872-510961d1201a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:27.968327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111573d1-0fd3-4398-8e9a-67d195e0288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:27.969211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111573d1-0fd3-4398-8e9a-67d195e0288a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:27.969489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111573d1-0fd3-4398-8e9a-67d195e0288a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:27.969761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111573d1-0fd3-4398-8e9a-67d195e0288a None admin] PATH_INFO: `/v3/users/d04590925e4941678119372638c4b9e3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:27.971989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111573d1-0fd3-4398-8e9a-67d195e0288a None admin] RBAC: Authorizing `identity:get_user(user_id=d04590925e4941678119372638c4b9e3)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:27.973479 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111573d1-0fd3-4398-8e9a-67d195e0288a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:27.975125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 98/200] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:27 2022] GET /identity/v3/users/d04590925e4941678119372638c4b9e3 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:27.975974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111573d1-0fd3-4398-8e9a-67d195e0288a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:28.017009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:28.029142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab75ed-1bbb-4f10-82de-7cbe19137e31 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:28.072827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e70044-18fd-4293-8c2f-a959a39fab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:28.073654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e70044-18fd-4293-8c2f-a959a39fab8e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:28.074886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e70044-18fd-4293-8c2f-a959a39fab8e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:28.075304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e70044-18fd-4293-8c2f-a959a39fab8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:28.093841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111573d1-0fd3-4398-8e9a-67d195e0288a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:28.140648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f14afa-3838-4597-9b97-cc725519fa8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:28.142378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f14afa-3838-4597-9b97-cc725519fa8e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:28.142901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f14afa-3838-4597-9b97-cc725519fa8e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:28.143340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f14afa-3838-4597-9b97-cc725519fa8e None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:28.145020 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f14afa-3838-4597-9b97-cc725519fa8e None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:28.147327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f14afa-3838-4597-9b97-cc725519fa8e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:28.149517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 99/201] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:27 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:28.151143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f14afa-3838-4597-9b97-cc725519fa8e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:28.153831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e70044-18fd-4293-8c2f-a959a39fab8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:28.155571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:28.157224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e70044-18fd-4293-8c2f-a959a39fab8e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:28.268150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:28.268752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f14afa-3838-4597-9b97-cc725519fa8e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:28.278954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 103/202] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:28.280068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e70044-18fd-4293-8c2f-a959a39fab8e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:28.305736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd869a-dfe1-4a23-8b51-adf62286fb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:28.306486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd869a-dfe1-4a23-8b51-adf62286fb25 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:28.306876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd869a-dfe1-4a23-8b51-adf62286fb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:28.307155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd869a-dfe1-4a23-8b51-adf62286fb25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:28.380382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd869a-dfe1-4a23-8b51-adf62286fb25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:28.381943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd869a-dfe1-4a23-8b51-adf62286fb25 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:28.384737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:28.391021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e70044-18fd-4293-8c2f-a959a39fab8e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:28.429274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a7d60-3337-4dab-b116-48fc1f7f6b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:28.430324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a7d60-3337-4dab-b116-48fc1f7f6b00 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:28.430825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a7d60-3337-4dab-b116-48fc1f7f6b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:28.431308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a7d60-3337-4dab-b116-48fc1f7f6b00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:28.432322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a7d60-3337-4dab-b116-48fc1f7f6b00 None admin] RBAC: Adding query filter params (role.id=feffc80bdc3641a59c6d5d545c7fce94, scope.project.id=380cd1f0793045d280c092a275d9666a, user.id=808c0899b68e44a28407207932e719fa) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:28.432804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a7d60-3337-4dab-b116-48fc1f7f6b00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:28.435314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a7d60-3337-4dab-b116-48fc1f7f6b00 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:28.442229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 104/203] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:28 2022] GET /identity/v3/role_assignments?user.id=808c0899b68e44a28407207932e719fa&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=feffc80bdc3641a59c6d5d545c7fce94 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:28.443225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a7d60-3337-4dab-b116-48fc1f7f6b00 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:28.510778 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:28.537037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 100/204] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:28.543544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd869a-dfe1-4a23-8b51-adf62286fb25 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:28.561151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a7d60-3337-4dab-b116-48fc1f7f6b00 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:28.599646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20162c48-e4c2-435f-a6c9-0fe2f3114a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:28.600819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20162c48-e4c2-435f-a6c9-0fe2f3114a83 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:28.601317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20162c48-e4c2-435f-a6c9-0fe2f3114a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:28.601791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20162c48-e4c2-435f-a6c9-0fe2f3114a83 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:28.603276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20162c48-e4c2-435f-a6c9-0fe2f3114a83 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:28.605555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20162c48-e4c2-435f-a6c9-0fe2f3114a83 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:28.608707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 105/205] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:28 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:28.614131 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20162c48-e4c2-435f-a6c9-0fe2f3114a83 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:28.735219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd869a-dfe1-4a23-8b51-adf62286fb25 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:28.738435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:28.775881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0ec54-4a16-44f0-a405-4e51394606f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:28.777358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0ec54-4a16-44f0-a405-4e51394606f1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:28.777735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0ec54-4a16-44f0-a405-4e51394606f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:28.777735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0ec54-4a16-44f0-a405-4e51394606f1 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:28.779931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0ec54-4a16-44f0-a405-4e51394606f1 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:28.782351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0ec54-4a16-44f0-a405-4e51394606f1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:28.784260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 101/206] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:28 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:28.789854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0ec54-4a16-44f0-a405-4e51394606f1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:28.809663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20162c48-e4c2-435f-a6c9-0fe2f3114a83 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:28.847667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:28.862542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944d522-8b42-47f9-b1d2-81aa7e432036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:28.863976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944d522-8b42-47f9-b1d2-81aa7e432036 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:28.864434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944d522-8b42-47f9-b1d2-81aa7e432036 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:28.864858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944d522-8b42-47f9-b1d2-81aa7e432036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:28.866299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944d522-8b42-47f9-b1d2-81aa7e432036 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca, user.id=d04590925e4941678119372638c4b9e3) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:28.867016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944d522-8b42-47f9-b1d2-81aa7e432036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:28.868640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944d522-8b42-47f9-b1d2-81aa7e432036 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:28.879414 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 106/207] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:28 2022] GET /identity/v3/role_assignments?user.id=d04590925e4941678119372638c4b9e3&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:28.926341 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0ec54-4a16-44f0-a405-4e51394606f1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:28.978511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67130faf-835b-43ba-ace4-acfe35d5572b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:28.979196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67130faf-835b-43ba-ace4-acfe35d5572b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:28.979478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67130faf-835b-43ba-ace4-acfe35d5572b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:28.979751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67130faf-835b-43ba-ace4-acfe35d5572b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:28.981479 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67130faf-835b-43ba-ace4-acfe35d5572b None admin] RBAC: Adding query filter params (role.id=110c20a9ff1e455d8c43ffdfbdd1447b, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca, user.id=d04590925e4941678119372638c4b9e3) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:28.981661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67130faf-835b-43ba-ace4-acfe35d5572b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:28.983080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67130faf-835b-43ba-ace4-acfe35d5572b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:28.989824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 102/208] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:28 2022] GET /identity/v3/role_assignments?user.id=d04590925e4941678119372638c4b9e3&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=110c20a9ff1e455d8c43ffdfbdd1447b => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:29.890419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e4ea-c8ee-4da1-92a8-6af326ac43fb None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:29.890959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e4ea-c8ee-4da1-92a8-6af326ac43fb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:29.890959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e4ea-c8ee-4da1-92a8-6af326ac43fb None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:29.895701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 107/209] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:29.895817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fbb5a-5fd1-4298-b8ff-c2286837bacc None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:29.895817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fbb5a-5fd1-4298-b8ff-c2286837bacc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:29.895817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fbb5a-5fd1-4298-b8ff-c2286837bacc None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:29.895817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 103/210] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:29.897802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c4906-fa52-4fe7-ac4b-7f66f9fa7b74 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:29.898101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c4906-fa52-4fe7-ac4b-7f66f9fa7b74 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:29.898382 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c4906-fa52-4fe7-ac4b-7f66f9fa7b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:29.902847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c27b-6f6e-49a5-91f9-5b4bbefe7398 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:29.903376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c27b-6f6e-49a5-91f9-5b4bbefe7398 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:29.903886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c27b-6f6e-49a5-91f9-5b4bbefe7398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:29.916733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-391c4906-fa52-4fe7-ac4b-7f66f9fa7b74 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:29.919115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-391c4906-fa52-4fe7-ac4b-7f66f9fa7b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:29.922604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-65e1c27b-6f6e-49a5-91f9-5b4bbefe7398 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:29.924026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-65e1c27b-6f6e-49a5-91f9-5b4bbefe7398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:29.956697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:29.996519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 108/211] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:29.999945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 104/212] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:30.015049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c4906-fa52-4fe7-ac4b-7f66f9fa7b74 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:30.024740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1c27b-6f6e-49a5-91f9-5b4bbefe7398 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:30.122469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c4906-fa52-4fe7-ac4b-7f66f9fa7b74 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:30.138161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1c27b-6f6e-49a5-91f9-5b4bbefe7398 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:30.140745 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:30.158247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91551e87-4ee6-4e6f-b23e-e91d45c72619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:30.159308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91551e87-4ee6-4e6f-b23e-e91d45c72619 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:30.159734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91551e87-4ee6-4e6f-b23e-e91d45c72619 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:30.160216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91551e87-4ee6-4e6f-b23e-e91d45c72619 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:30.161606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91551e87-4ee6-4e6f-b23e-e91d45c72619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:30.163314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91551e87-4ee6-4e6f-b23e-e91d45c72619 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:30.165189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 109/213] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:30.171465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91551e87-4ee6-4e6f-b23e-e91d45c72619 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:30.178368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947db19-c866-45f2-a21a-92a576564eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:30.179181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947db19-c866-45f2-a21a-92a576564eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:30.179465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947db19-c866-45f2-a21a-92a576564eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:30.179742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947db19-c866-45f2-a21a-92a576564eb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:30.192220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947db19-c866-45f2-a21a-92a576564eb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6947db19-c866-45f2-a21a-92a576564eb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:30.206009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:30.206879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:30.208070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:30.212309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 105/214] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:30.214095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947db19-c866-45f2-a21a-92a576564eb8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:30.252247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:30.286807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91551e87-4ee6-4e6f-b23e-e91d45c72619 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:30.323821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae96428-555a-4fe5-9837-8862dcd9260b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:30.325108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae96428-555a-4fe5-9837-8862dcd9260b None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:30.325670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae96428-555a-4fe5-9837-8862dcd9260b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:30.326200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae96428-555a-4fe5-9837-8862dcd9260b None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:30.327160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae96428-555a-4fe5-9837-8862dcd9260b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:30.328851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae96428-555a-4fe5-9837-8862dcd9260b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:30.351607 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947db19-c866-45f2-a21a-92a576564eb8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:30.354034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 110/215] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:20:30 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:20:30.404839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce65ad5-7d69-4337-9445-52909f0ea2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:30.405438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce65ad5-7d69-4337-9445-52909f0ea2af None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:30.405774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce65ad5-7d69-4337-9445-52909f0ea2af None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:30.406058 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce65ad5-7d69-4337-9445-52909f0ea2af None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:30.406935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce65ad5-7d69-4337-9445-52909f0ea2af None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:30.407253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce65ad5-7d69-4337-9445-52909f0ea2af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:30.408741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce65ad5-7d69-4337-9445-52909f0ea2af None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:30.418881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 106/216] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:30.424550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae96428-555a-4fe5-9837-8862dcd9260b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:30.477209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:30.542334 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae96428-555a-4fe5-9837-8862dcd9260b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:30.577727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24f9ea-39f0-4fed-8ab4-83dd63b4d006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:30.578666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24f9ea-39f0-4fed-8ab4-83dd63b4d006 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:30.578946 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24f9ea-39f0-4fed-8ab4-83dd63b4d006 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:30.579214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24f9ea-39f0-4fed-8ab4-83dd63b4d006 None admin] PATH_INFO: `/v3/users/23b0c52acfdb4d4a81b1be970cda509b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:30.594136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24f9ea-39f0-4fed-8ab4-83dd63b4d006 None admin] RBAC: Authorizing `identity:get_user(user_id=23b0c52acfdb4d4a81b1be970cda509b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:30.595552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24f9ea-39f0-4fed-8ab4-83dd63b4d006 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:30.597567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 111/217] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:30 2022] GET /identity/v3/users/23b0c52acfdb4d4a81b1be970cda509b => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:30.603394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce65ad5-7d69-4337-9445-52909f0ea2af None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:30.701622 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:30.716215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce65ad5-7d69-4337-9445-52909f0ea2af None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:30.752701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3a508-06c3-4818-aff1-b95a506e3468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:30.753357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3a508-06c3-4818-aff1-b95a506e3468 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:30.753637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3a508-06c3-4818-aff1-b95a506e3468 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:30.753915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3a508-06c3-4818-aff1-b95a506e3468 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:30.827982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3a508-06c3-4818-aff1-b95a506e3468 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:30.829478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3a508-06c3-4818-aff1-b95a506e3468 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:30.928774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:30.949881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 107/218] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:30.954994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24f9ea-39f0-4fed-8ab4-83dd63b4d006 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:31.066379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffb23b-147c-4827-b5de-ac727eeccd0b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:31.066694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffb23b-147c-4827-b5de-ac727eeccd0b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:31.066947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffb23b-147c-4827-b5de-ac727eeccd0b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:31.068499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 108/219] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:31.074789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae2a1d-1bae-44f7-9756-db045cf15739 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:31.075415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae2a1d-1bae-44f7-9756-db045cf15739 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:31.075883 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae2a1d-1bae-44f7-9756-db045cf15739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:31.094331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4fae2a1d-1bae-44f7-9756-db045cf15739 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:31.096505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4fae2a1d-1bae-44f7-9756-db045cf15739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:31.112025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:31.116450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24f9ea-39f0-4fed-8ab4-83dd63b4d006 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:31.154950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8761336-624e-4f26-8bff-00494ec6fa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:31.156238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8761336-624e-4f26-8bff-00494ec6fa9b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:31.156742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8761336-624e-4f26-8bff-00494ec6fa9b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:31.157208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8761336-624e-4f26-8bff-00494ec6fa9b None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:31.158660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8761336-624e-4f26-8bff-00494ec6fa9b None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:31.160395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8761336-624e-4f26-8bff-00494ec6fa9b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:31.162844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 112/220] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:30 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:31.164261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd240e01-9757-4dab-bf37-3cd8a9df2771 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:31.169967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd240e01-9757-4dab-bf37-3cd8a9df2771 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:31.170723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd240e01-9757-4dab-bf37-3cd8a9df2771 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:31.172568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 113/221] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:31.173608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd240e01-9757-4dab-bf37-3cd8a9df2771 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:31.184808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 109/222] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:31.186098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f1d87-70c8-4b8c-b9c6-9c01c70b09f1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:31.186384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f1d87-70c8-4b8c-b9c6-9c01c70b09f1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:31.186658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f1d87-70c8-4b8c-b9c6-9c01c70b09f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:31.218875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-837f1d87-70c8-4b8c-b9c6-9c01c70b09f1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:31.220414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-837f1d87-70c8-4b8c-b9c6-9c01c70b09f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:31.291683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 110/223] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:31.300697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f1d87-70c8-4b8c-b9c6-9c01c70b09f1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:31.302390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd240e01-9757-4dab-bf37-3cd8a9df2771 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:31.342438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6843d37-a597-4880-8b79-effc621c86bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:31.343771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6843d37-a597-4880-8b79-effc621c86bc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:31.344572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6843d37-a597-4880-8b79-effc621c86bc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:31.345503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6843d37-a597-4880-8b79-effc621c86bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:31.346877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6843d37-a597-4880-8b79-effc621c86bc None admin] RBAC: Adding query filter params (role.id=d2c2845de1d548bb9cfb46f70619b3f8, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca, user.id=23b0c52acfdb4d4a81b1be970cda509b) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:31.348999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6843d37-a597-4880-8b79-effc621c86bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:31.351831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6843d37-a597-4880-8b79-effc621c86bc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:31.359415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 114/224] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:31 2022] GET /identity/v3/role_assignments?user.id=23b0c52acfdb4d4a81b1be970cda509b&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:31.369650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6843d37-a597-4880-8b79-effc621c86bc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:31.370300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:31.425582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:31.437953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f1d87-70c8-4b8c-b9c6-9c01c70b09f1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:31.481498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca049b1-b3a1-4f24-abdf-d7ee612baf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:31.482332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca049b1-b3a1-4f24-abdf-d7ee612baf73 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:31.482621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca049b1-b3a1-4f24-abdf-d7ee612baf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:31.482899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca049b1-b3a1-4f24-abdf-d7ee612baf73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:31.497724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca049b1-b3a1-4f24-abdf-d7ee612baf73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-4ca049b1-b3a1-4f24-abdf-d7ee612baf73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:31.506484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:31.507414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:31.508203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 111/225] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:31.514978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca049b1-b3a1-4f24-abdf-d7ee612baf73 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:31.578370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6843d37-a597-4880-8b79-effc621c86bc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:31.644367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca049b1-b3a1-4f24-abdf-d7ee612baf73 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:31.657151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:31.657554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d8d97-3de3-4c82-a091-c64bc2ebd9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:31.659428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8d97-3de3-4c82-a091-c64bc2ebd9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:31.659921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8d97-3de3-4c82-a091-c64bc2ebd9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:31.660531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8d97-3de3-4c82-a091-c64bc2ebd9e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:31.681972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bd9b7-4482-4a6e-8efe-20204abc122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:31.682654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd9b7-4482-4a6e-8efe-20204abc122e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:31.682942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd9b7-4482-4a6e-8efe-20204abc122e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:31.683439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd9b7-4482-4a6e-8efe-20204abc122e None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:31.684272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bd9b7-4482-4a6e-8efe-20204abc122e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:31.684720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bd9b7-4482-4a6e-8efe-20204abc122e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:31.686892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bd9b7-4482-4a6e-8efe-20204abc122e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:31.695859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 112/226] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:31.701202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d8d97-3de3-4c82-a091-c64bc2ebd9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:31.702224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bd9b7-4482-4a6e-8efe-20204abc122e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-242d8d97-3de3-4c82-a091-c64bc2ebd9e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:31.717833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:31.718756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:31.719541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 115/227] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:31.726460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d8d97-3de3-4c82-a091-c64bc2ebd9e6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:31.809805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bd9b7-4482-4a6e-8efe-20204abc122e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:31.845869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d7545-c444-41ba-8733-a19479d76b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:31.846765 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d7545-c444-41ba-8733-a19479d76b22 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:31.847060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d7545-c444-41ba-8733-a19479d76b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:31.847347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d7545-c444-41ba-8733-a19479d76b22 None admin] PATH_INFO: `/v3/users/d04590925e4941678119372638c4b9e3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:31.848613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d7545-c444-41ba-8733-a19479d76b22 None admin] RBAC: Authorizing `identity:get_user(user_id=d04590925e4941678119372638c4b9e3)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:31.850219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d7545-c444-41ba-8733-a19479d76b22 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:31.851883 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 113/228] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:31 2022] GET /identity/v3/users/d04590925e4941678119372638c4b9e3 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:31.857368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d7545-c444-41ba-8733-a19479d76b22 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:31.932776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:31.967484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d7545-c444-41ba-8733-a19479d76b22 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:32.003472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389aaf1e-ba87-4537-9a04-6768437ef8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:32.004325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389aaf1e-ba87-4537-9a04-6768437ef8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:32.004620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389aaf1e-ba87-4537-9a04-6768437ef8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:32.004901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389aaf1e-ba87-4537-9a04-6768437ef8e6 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:32.006114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389aaf1e-ba87-4537-9a04-6768437ef8e6 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:32.007568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389aaf1e-ba87-4537-9a04-6768437ef8e6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:32.009499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 114/229] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:31 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:32.015643 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389aaf1e-ba87-4537-9a04-6768437ef8e6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:32.021432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:32.066334 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d8d97-3de3-4c82-a091-c64bc2ebd9e6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:32.122290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389aaf1e-ba87-4537-9a04-6768437ef8e6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:32.145562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce156d-bd72-4b93-9df3-cdfeea03e772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:32.146253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce156d-bd72-4b93-9df3-cdfeea03e772 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:32.146542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce156d-bd72-4b93-9df3-cdfeea03e772 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:32.146825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce156d-bd72-4b93-9df3-cdfeea03e772 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:32.147592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce156d-bd72-4b93-9df3-cdfeea03e772 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:32.147902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce156d-bd72-4b93-9df3-cdfeea03e772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:32.149363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce156d-bd72-4b93-9df3-cdfeea03e772 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:32.152745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:32.158601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877be4d-8867-45f4-9931-2ac29b9c8f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:32.159505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877be4d-8867-45f4-9931-2ac29b9c8f7d None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:32.159843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877be4d-8867-45f4-9931-2ac29b9c8f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:32.160085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877be4d-8867-45f4-9931-2ac29b9c8f7d None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/users/d04590925e4941678119372638c4b9e3/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:32.162216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877be4d-8867-45f4-9931-2ac29b9c8f7d None admin] RBAC: Authorizing `identity:create_grant(project_id=4819e5a17eb24efcb8ccbb51a48b30ca, user_id=d04590925e4941678119372638c4b9e3, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:32.163785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877be4d-8867-45f4-9931-2ac29b9c8f7d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:32.172149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 116/230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:32.174211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce156d-bd72-4b93-9df3-cdfeea03e772 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:32.192450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 115/231] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:20:32 2022] PUT /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/users/d04590925e4941678119372638c4b9e3/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:32.326619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce156d-bd72-4b93-9df3-cdfeea03e772 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:32.374534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67a156-efec-4951-95af-7da911751e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:32.375463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67a156-efec-4951-95af-7da911751e9e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:32.375750 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67a156-efec-4951-95af-7da911751e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:32.376029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67a156-efec-4951-95af-7da911751e9e None admin] PATH_INFO: `/v3/users/d04590925e4941678119372638c4b9e3` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:32.377306 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67a156-efec-4951-95af-7da911751e9e None admin] RBAC: Authorizing `identity:get_user(user_id=d04590925e4941678119372638c4b9e3)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:32.378805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67a156-efec-4951-95af-7da911751e9e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:32.380717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 117/232] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:32 2022] GET /identity/v3/users/d04590925e4941678119372638c4b9e3 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:32.386052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877be4d-8867-45f4-9931-2ac29b9c8f7d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:32.510081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877be4d-8867-45f4-9931-2ac29b9c8f7d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:32.546725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8a721-98aa-45fb-a964-3f0fe53b1740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:32.547602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8a721-98aa-45fb-a964-3f0fe53b1740 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:32.547884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8a721-98aa-45fb-a964-3f0fe53b1740 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:32.548159 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8a721-98aa-45fb-a964-3f0fe53b1740 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:32.549398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8a721-98aa-45fb-a964-3f0fe53b1740 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:32.550853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8a721-98aa-45fb-a964-3f0fe53b1740 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:32.552906 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 116/233] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:32 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:32.558311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67a156-efec-4951-95af-7da911751e9e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:32.602574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:32.703124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67a156-efec-4951-95af-7da911751e9e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:32.743802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b302f4-24ab-4ba2-91c2-a4980f55a1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:32.744738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b302f4-24ab-4ba2-91c2-a4980f55a1f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:32.745025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b302f4-24ab-4ba2-91c2-a4980f55a1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:32.745305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b302f4-24ab-4ba2-91c2-a4980f55a1f3 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/users/d04590925e4941678119372638c4b9e3/roles/110c20a9ff1e455d8c43ffdfbdd1447b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:32.747650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b302f4-24ab-4ba2-91c2-a4980f55a1f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4819e5a17eb24efcb8ccbb51a48b30ca, user_id=d04590925e4941678119372638c4b9e3, role_id=110c20a9ff1e455d8c43ffdfbdd1447b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:32.749141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b302f4-24ab-4ba2-91c2-a4980f55a1f3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:32.769633 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 118/234] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:20:32 2022] PUT /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/users/d04590925e4941678119372638c4b9e3/roles/110c20a9ff1e455d8c43ffdfbdd1447b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:33.006384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1e4e5-05ec-454b-9764-65da25531f47 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:33.006885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1e4e5-05ec-454b-9764-65da25531f47 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:33.006958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1e4e5-05ec-454b-9764-65da25531f47 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:33.008488 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 117/235] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:33.014431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0be88-d913-4c0c-8b40-99ae62539f7a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:33.014725 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0be88-d913-4c0c-8b40-99ae62539f7a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:33.015009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0be88-d913-4c0c-8b40-99ae62539f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:33.034864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-26e0be88-d913-4c0c-8b40-99ae62539f7a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:33.036089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-26e0be88-d913-4c0c-8b40-99ae62539f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:33.115956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 119/236] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:33.153705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1e4e5-05ec-454b-9764-65da25531f47 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:33.202849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:33.285986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1e4e5-05ec-454b-9764-65da25531f47 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:33.329950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94846005-cc02-408e-9753-2a278a31b01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:33.330762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94846005-cc02-408e-9753-2a278a31b01d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:33.331047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94846005-cc02-408e-9753-2a278a31b01d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:33.331326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94846005-cc02-408e-9753-2a278a31b01d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:33.345622 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94846005-cc02-408e-9753-2a278a31b01d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-94846005-cc02-408e-9753-2a278a31b01d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:33.353478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:33.354331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:33.356435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 118/237] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:33.361804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602eda-d328-4206-b239-4626cb723bde None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:33.362098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602eda-d328-4206-b239-4626cb723bde None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:33.362383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602eda-d328-4206-b239-4626cb723bde None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:33.364005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 120/238] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:33.364197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94846005-cc02-408e-9753-2a278a31b01d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:33.369575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af927f8-bae2-402a-9ee0-ed528bc0cce8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:33.369866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af927f8-bae2-402a-9ee0-ed528bc0cce8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:33.370148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af927f8-bae2-402a-9ee0-ed528bc0cce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:33.408942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6af927f8-bae2-402a-9ee0-ed528bc0cce8 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:33.410474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6af927f8-bae2-402a-9ee0-ed528bc0cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:33.425537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:33.479910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:33.487068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94846005-cc02-408e-9753-2a278a31b01d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:33.545707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112ba81-f131-4a61-946e-fb0a56c2aacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:33.546984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112ba81-f131-4a61-946e-fb0a56c2aacf None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:33.547281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112ba81-f131-4a61-946e-fb0a56c2aacf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:33.547563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112ba81-f131-4a61-946e-fb0a56c2aacf None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:33.548337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112ba81-f131-4a61-946e-fb0a56c2aacf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:33.548654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112ba81-f131-4a61-946e-fb0a56c2aacf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:33.550031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112ba81-f131-4a61-946e-fb0a56c2aacf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:33.554889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 121/239] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:33.569031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 119/240] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:33.575148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af927f8-bae2-402a-9ee0-ed528bc0cce8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:33.622039 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112ba81-f131-4a61-946e-fb0a56c2aacf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:33.762508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af927f8-bae2-402a-9ee0-ed528bc0cce8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:33.803480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112ba81-f131-4a61-946e-fb0a56c2aacf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:33.828146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb61ef-3816-4b91-b476-88cc93952974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:33.829163 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb61ef-3816-4b91-b476-88cc93952974 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:33.829457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb61ef-3816-4b91-b476-88cc93952974 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:33.829733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb61ef-3816-4b91-b476-88cc93952974 None admin] PATH_INFO: `/v3/users/ee7220c334af420291c478e2dd299667` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:33.842266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:33.846048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb61ef-3816-4b91-b476-88cc93952974 None admin] RBAC: Authorizing `identity:get_user(user_id=ee7220c334af420291c478e2dd299667)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:33.848058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb61ef-3816-4b91-b476-88cc93952974 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:33.850174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 122/241] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:33 2022] GET /identity/v3/users/ee7220c334af420291c478e2dd299667 => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:33.851485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8e88-8c05-4c78-89f0-3cd244968ded None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:33.851779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8e88-8c05-4c78-89f0-3cd244968ded None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:33.852060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8e88-8c05-4c78-89f0-3cd244968ded None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:33.852672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dbfde-2954-4166-ae15-bdadd800e581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:33.853328 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 123/242] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:33.853911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dbfde-2954-4166-ae15-bdadd800e581 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:33.854393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dbfde-2954-4166-ae15-bdadd800e581 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:33.855395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dbfde-2954-4166-ae15-bdadd800e581 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:33.856843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dbfde-2954-4166-ae15-bdadd800e581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:33.858491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dbfde-2954-4166-ae15-bdadd800e581 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:33.858800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e8e88-8c05-4c78-89f0-3cd244968ded None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:33.860593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 120/243] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:33.862083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13307518-e1e0-445d-a4ce-62f255943900 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:33.862693 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13307518-e1e0-445d-a4ce-62f255943900 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:33.863256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13307518-e1e0-445d-a4ce-62f255943900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:33.880699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-13307518-e1e0-445d-a4ce-62f255943900 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:33.882223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-13307518-e1e0-445d-a4ce-62f255943900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:33.901218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:33.966783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 121/244] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:33.968158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13307518-e1e0-445d-a4ce-62f255943900 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:33.999263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e8e88-8c05-4c78-89f0-3cd244968ded None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:34.036092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebea302-f2c2-4dbf-9d08-b6631ca14c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:34.036760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebea302-f2c2-4dbf-9d08-b6631ca14c1e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:34.036956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebea302-f2c2-4dbf-9d08-b6631ca14c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:34.037403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebea302-f2c2-4dbf-9d08-b6631ca14c1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:34.102880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13307518-e1e0-445d-a4ce-62f255943900 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:34.112717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:34.115592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebea302-f2c2-4dbf-9d08-b6631ca14c1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:34.117496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebea302-f2c2-4dbf-9d08-b6631ca14c1e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:34.148452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9785c98-3059-4940-bafc-79dc945a659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:34.149371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9785c98-3059-4940-bafc-79dc945a659a None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:34.149801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9785c98-3059-4940-bafc-79dc945a659a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:34.150221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9785c98-3059-4940-bafc-79dc945a659a None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:34.151145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9785c98-3059-4940-bafc-79dc945a659a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:34.153365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9785c98-3059-4940-bafc-79dc945a659a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:34.176642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 122/245] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:20:33 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:34.178104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9785c98-3059-4940-bafc-79dc945a659a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:34.194179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:34.242496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 124/246] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:34.248171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebea302-f2c2-4dbf-9d08-b6631ca14c1e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:34.301480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9785c98-3059-4940-bafc-79dc945a659a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:34.341092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b96f7-8c07-44b2-bfec-8fbeb7f2b907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:34.342536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b96f7-8c07-44b2-bfec-8fbeb7f2b907 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:34.343030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b96f7-8c07-44b2-bfec-8fbeb7f2b907 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:34.343493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b96f7-8c07-44b2-bfec-8fbeb7f2b907 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:34.357602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b96f7-8c07-44b2-bfec-8fbeb7f2b907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:34.361231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:34.364901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebea302-f2c2-4dbf-9d08-b6631ca14c1e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-2e1b96f7-8c07-44b2-bfec-8fbeb7f2b907 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:34.367678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:34.368635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:34.369432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:34.376424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 123/247] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:34.378450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b96f7-8c07-44b2-bfec-8fbeb7f2b907 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:34.406104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba0ec-10eb-439a-bddb-225d576e4c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:34.407076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba0ec-10eb-439a-bddb-225d576e4c2c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:34.407361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba0ec-10eb-439a-bddb-225d576e4c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:34.407634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba0ec-10eb-439a-bddb-225d576e4c2c None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:34.409018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeba0ec-10eb-439a-bddb-225d576e4c2c None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:34.410677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeba0ec-10eb-439a-bddb-225d576e4c2c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:34.412928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 125/248] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:34 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:34.418930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba0ec-10eb-439a-bddb-225d576e4c2c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:34.481696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:34.519418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b96f7-8c07-44b2-bfec-8fbeb7f2b907 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:34.558047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba0ec-10eb-439a-bddb-225d576e4c2c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:34.570583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e57107-8027-464a-9390-15c9216109ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:34.571268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e57107-8027-464a-9390-15c9216109ec None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:34.571551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e57107-8027-464a-9390-15c9216109ec None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:34.571827 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e57107-8027-464a-9390-15c9216109ec None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:34.573033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e57107-8027-464a-9390-15c9216109ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:34.573346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e57107-8027-464a-9390-15c9216109ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:34.576073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e57107-8027-464a-9390-15c9216109ec None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:34.588101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 124/249] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:34.591387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1578a40-ecc5-4728-a3bb-ad3b4e0ba8d3 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:34.591387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1578a40-ecc5-4728-a3bb-ad3b4e0ba8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:34.591550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1578a40-ecc5-4728-a3bb-ad3b4e0ba8d3 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:34.597601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 125/250] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:34.602136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544935c-8abd-46fa-863a-ba6775483f20 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:34.602398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544935c-8abd-46fa-863a-ba6775483f20 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:34.602683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544935c-8abd-46fa-863a-ba6775483f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:34.607736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5902648-d7b4-4e63-b1a5-be0357f4a084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:34.608645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902648-d7b4-4e63-b1a5-be0357f4a084 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:34.609758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902648-d7b4-4e63-b1a5-be0357f4a084 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:34.610050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902648-d7b4-4e63-b1a5-be0357f4a084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:34.610893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5902648-d7b4-4e63-b1a5-be0357f4a084 None admin] RBAC: Adding query filter params (role.id=feffc80bdc3641a59c6d5d545c7fce94, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca, user.id=ee7220c334af420291c478e2dd299667) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:34.611208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5902648-d7b4-4e63-b1a5-be0357f4a084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:34.612726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5902648-d7b4-4e63-b1a5-be0357f4a084 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:34.619561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 126/251] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:34 2022] GET /identity/v3/role_assignments?user.id=ee7220c334af420291c478e2dd299667&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=feffc80bdc3641a59c6d5d545c7fce94 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:34.622851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5902648-d7b4-4e63-b1a5-be0357f4a084 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:34.625313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4544935c-8abd-46fa-863a-ba6775483f20 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:34.626767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4544935c-8abd-46fa-863a-ba6775483f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:34.666780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:34.706580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 126/252] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:34.737206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544935c-8abd-46fa-863a-ba6775483f20 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:34.759815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5902648-d7b4-4e63-b1a5-be0357f4a084 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:34.800104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75451c-b2ea-4776-9fd9-f5c230fb5cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:34.801204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75451c-b2ea-4776-9fd9-f5c230fb5cdb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:34.801340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75451c-b2ea-4776-9fd9-f5c230fb5cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:34.801620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75451c-b2ea-4776-9fd9-f5c230fb5cdb None admin] PATH_INFO: `/v3/users/23b0c52acfdb4d4a81b1be970cda509b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:34.802876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75451c-b2ea-4776-9fd9-f5c230fb5cdb None admin] RBAC: Authorizing `identity:get_user(user_id=23b0c52acfdb4d4a81b1be970cda509b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:34.804426 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75451c-b2ea-4776-9fd9-f5c230fb5cdb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:34.806119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 127/253] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:34 2022] GET /identity/v3/users/23b0c52acfdb4d4a81b1be970cda509b => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:34.814236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75451c-b2ea-4776-9fd9-f5c230fb5cdb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:34.828369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:34.877464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544935c-8abd-46fa-863a-ba6775483f20 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:34.938507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57162c93-ec92-48c6-b4a1-673b5ebc22ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:34.939656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57162c93-ec92-48c6-b4a1-673b5ebc22ad None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:34.939939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57162c93-ec92-48c6-b4a1-673b5ebc22ad None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:34.940212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57162c93-ec92-48c6-b4a1-673b5ebc22ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:34.945796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75451c-b2ea-4776-9fd9-f5c230fb5cdb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:34.954937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57162c93-ec92-48c6-b4a1-673b5ebc22ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:34.956723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-57162c93-ec92-48c6-b4a1-673b5ebc22ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:34.965900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:34.966800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:34.967604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 127/254] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:34.976339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57162c93-ec92-48c6-b4a1-673b5ebc22ad None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:35.024064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89766c8c-1a1e-486e-a8cd-187fb9609168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:35.025232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89766c8c-1a1e-486e-a8cd-187fb9609168 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:35.025656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89766c8c-1a1e-486e-a8cd-187fb9609168 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:35.026071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89766c8c-1a1e-486e-a8cd-187fb9609168 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:35.028228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89766c8c-1a1e-486e-a8cd-187fb9609168 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:35.031033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89766c8c-1a1e-486e-a8cd-187fb9609168 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:35.033675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 128/255] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:34 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:35.041218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89766c8c-1a1e-486e-a8cd-187fb9609168 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:35.175643 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57162c93-ec92-48c6-b4a1-673b5ebc22ad None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:35.204164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89766c8c-1a1e-486e-a8cd-187fb9609168 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:35.233993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:35.244009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55d46e-0f04-463a-9f79-4a85e6298c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:35.245269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d46e-0f04-463a-9f79-4a85e6298c91 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:35.246419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d46e-0f04-463a-9f79-4a85e6298c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:35.246905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d46e-0f04-463a-9f79-4a85e6298c91 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/users/23b0c52acfdb4d4a81b1be970cda509b/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:35.249184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55d46e-0f04-463a-9f79-4a85e6298c91 None admin] RBAC: Authorizing `identity:create_grant(project_id=4819e5a17eb24efcb8ccbb51a48b30ca, user_id=23b0c52acfdb4d4a81b1be970cda509b, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:35.250847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55d46e-0f04-463a-9f79-4a85e6298c91 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:35.261705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b8114-9bb0-4f00-add9-be7f7bfd3e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:35.262502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b8114-9bb0-4f00-add9-be7f7bfd3e48 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:35.262846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b8114-9bb0-4f00-add9-be7f7bfd3e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:35.263210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b8114-9bb0-4f00-add9-be7f7bfd3e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:35.264245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b8114-9bb0-4f00-add9-be7f7bfd3e48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:35.264583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b8114-9bb0-4f00-add9-be7f7bfd3e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:35.266102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b8114-9bb0-4f00-add9-be7f7bfd3e48 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:35.271867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 129/256] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:20:35 2022] PUT /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/users/23b0c52acfdb4d4a81b1be970cda509b/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:35.286734 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 128/257] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:35.291337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55d46e-0f04-463a-9f79-4a85e6298c91 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:35.355705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:35.427240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55d46e-0f04-463a-9f79-4a85e6298c91 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:35.468013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db5002-5a73-4037-8804-bed2e34bf6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:35.469070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db5002-5a73-4037-8804-bed2e34bf6ea None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:35.469353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db5002-5a73-4037-8804-bed2e34bf6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:35.469631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db5002-5a73-4037-8804-bed2e34bf6ea None admin] PATH_INFO: `/v3/users/d04590925e4941678119372638c4b9e3` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:35.470843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db5002-5a73-4037-8804-bed2e34bf6ea None admin] RBAC: Authorizing `identity:get_user(user_id=d04590925e4941678119372638c4b9e3)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:35.472444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db5002-5a73-4037-8804-bed2e34bf6ea None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:35.474478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 130/258] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:35 2022] GET /identity/v3/users/d04590925e4941678119372638c4b9e3 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:35.480214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b8114-9bb0-4f00-add9-be7f7bfd3e48 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:35.484824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627aee60-9954-4372-bf74-697f44c344f0 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:35.485113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627aee60-9954-4372-bf74-697f44c344f0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:35.485391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627aee60-9954-4372-bf74-697f44c344f0 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:35.486893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 131/259] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:35.502034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14074a5-38da-4252-a54f-2274701ca278 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:35.502332 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14074a5-38da-4252-a54f-2274701ca278 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:35.502594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14074a5-38da-4252-a54f-2274701ca278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:35.529204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e14074a5-38da-4252-a54f-2274701ca278 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:35.530464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e14074a5-38da-4252-a54f-2274701ca278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:35.595648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b8114-9bb0-4f00-add9-be7f7bfd3e48 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:35.643282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee104a1-3b27-4116-aeb0-edcb6576dc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:35.643951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee104a1-3b27-4116-aeb0-edcb6576dc07 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:35.644230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee104a1-3b27-4116-aeb0-edcb6576dc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:35.646335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee104a1-3b27-4116-aeb0-edcb6576dc07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:35.669314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 132/260] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:35.720737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:35.743630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee104a1-3b27-4116-aeb0-edcb6576dc07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:35.745144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee104a1-3b27-4116-aeb0-edcb6576dc07 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:35.754454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14074a5-38da-4252-a54f-2274701ca278 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:35.800474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:35.978815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 129/261] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:35.983594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee104a1-3b27-4116-aeb0-edcb6576dc07 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:35.987307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14074a5-38da-4252-a54f-2274701ca278 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:36.055664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9f385-bbd5-4c1c-9228-740c079c05d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:36.056555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9f385-bbd5-4c1c-9228-740c079c05d6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:36.056835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9f385-bbd5-4c1c-9228-740c079c05d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:36.057110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9f385-bbd5-4c1c-9228-740c079c05d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:36.080592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9f385-bbd5-4c1c-9228-740c079c05d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6cd9f385-bbd5-4c1c-9228-740c079c05d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:36.088654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:36.089581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:36.091024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 133/262] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:36.098368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9f385-bbd5-4c1c-9228-740c079c05d6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:36.124651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:36.144619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee104a1-3b27-4116-aeb0-edcb6576dc07 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:36.205828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b05d7a-4e03-4c3b-8d35-ad6c76457f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:36.206703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b05d7a-4e03-4c3b-8d35-ad6c76457f51 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:36.207006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b05d7a-4e03-4c3b-8d35-ad6c76457f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:36.207245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b05d7a-4e03-4c3b-8d35-ad6c76457f51 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:36.212318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b05d7a-4e03-4c3b-8d35-ad6c76457f51 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:36.212318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b05d7a-4e03-4c3b-8d35-ad6c76457f51 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:36.212626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 130/263] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:35 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:36.218149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b05d7a-4e03-4c3b-8d35-ad6c76457f51 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:36.304747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:36.338670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9f385-bbd5-4c1c-9228-740c079c05d6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:36.410369 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465a6a9-57ea-4761-8c46-9324e54af704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:36.411055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465a6a9-57ea-4761-8c46-9324e54af704 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:36.411337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465a6a9-57ea-4761-8c46-9324e54af704 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:36.411613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465a6a9-57ea-4761-8c46-9324e54af704 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:36.412397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465a6a9-57ea-4761-8c46-9324e54af704 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:36.412707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465a6a9-57ea-4761-8c46-9324e54af704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:36.414124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465a6a9-57ea-4761-8c46-9324e54af704 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:36.429760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 134/264] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:36.439488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465a6a9-57ea-4761-8c46-9324e54af704 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:36.445894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b05d7a-4e03-4c3b-8d35-ad6c76457f51 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:36.533573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83ba11-6120-4b41-99c8-186ad28165c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:36.534250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83ba11-6120-4b41-99c8-186ad28165c0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:36.534546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83ba11-6120-4b41-99c8-186ad28165c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:36.534828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83ba11-6120-4b41-99c8-186ad28165c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:36.535676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83ba11-6120-4b41-99c8-186ad28165c0 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca, user.id=d04590925e4941678119372638c4b9e3) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:36.535990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83ba11-6120-4b41-99c8-186ad28165c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:36.537475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83ba11-6120-4b41-99c8-186ad28165c0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:36.554934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 131/265] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:36 2022] GET /identity/v3/role_assignments?user.id=d04590925e4941678119372638c4b9e3&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 608 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:36.582716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465a6a9-57ea-4761-8c46-9324e54af704 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:36.637829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e8ae3-87de-4e06-8a26-d2baffafcc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:36.638728 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e8ae3-87de-4e06-8a26-d2baffafcc7f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:36.639014 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e8ae3-87de-4e06-8a26-d2baffafcc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:36.639292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e8ae3-87de-4e06-8a26-d2baffafcc7f None admin] PATH_INFO: `/v3/users/d04590925e4941678119372638c4b9e3` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:36.640533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e8ae3-87de-4e06-8a26-d2baffafcc7f None admin] RBAC: Authorizing `identity:get_user(user_id=d04590925e4941678119372638c4b9e3)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:36.642147 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e8ae3-87de-4e06-8a26-d2baffafcc7f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:36.643770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 135/266] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:36 2022] GET /identity/v3/users/d04590925e4941678119372638c4b9e3 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:36.649540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83ba11-6120-4b41-99c8-186ad28165c0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:36.684751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:36.807313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83ba11-6120-4b41-99c8-186ad28165c0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:36.854337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e5562-4220-42be-af09-bd9739a4f231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:36.855091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5562-4220-42be-af09-bd9739a4f231 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:36.855375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5562-4220-42be-af09-bd9739a4f231 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:36.855651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5562-4220-42be-af09-bd9739a4f231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:36.957872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e5562-4220-42be-af09-bd9739a4f231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:36.959359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e5562-4220-42be-af09-bd9739a4f231 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:36.984005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:37.070049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 132/267] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:37.076626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e8ae3-87de-4e06-8a26-d2baffafcc7f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:37.096566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:37.190045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e8ae3-87de-4e06-8a26-d2baffafcc7f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:37.250443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22a114-347d-4610-ab26-e9cee5e07ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:37.251851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22a114-347d-4610-ab26-e9cee5e07ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:37.252617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22a114-347d-4610-ab26-e9cee5e07ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:37.253062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22a114-347d-4610-ab26-e9cee5e07ae7 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:37.257422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22a114-347d-4610-ab26-e9cee5e07ae7 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:37.260007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22a114-347d-4610-ab26-e9cee5e07ae7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:37.262712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 136/268] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:37 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:37.270814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e5562-4220-42be-af09-bd9739a4f231 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:37.458514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e5562-4220-42be-af09-bd9739a4f231 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:37.477281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:37.508046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f135b62-d74b-4fe3-85bb-006f5cd3c7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:37.508752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f135b62-d74b-4fe3-85bb-006f5cd3c7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:37.509130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f135b62-d74b-4fe3-85bb-006f5cd3c7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:37.509403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f135b62-d74b-4fe3-85bb-006f5cd3c7e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:37.510308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f135b62-d74b-4fe3-85bb-006f5cd3c7e3 None admin] RBAC: Adding query filter params (role.id=110c20a9ff1e455d8c43ffdfbdd1447b, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca, user.id=d04590925e4941678119372638c4b9e3) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:37.510617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f135b62-d74b-4fe3-85bb-006f5cd3c7e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:37.512962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f135b62-d74b-4fe3-85bb-006f5cd3c7e3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:37.519858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 133/269] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:37 2022] GET /identity/v3/role_assignments?user.id=d04590925e4941678119372638c4b9e3&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=110c20a9ff1e455d8c43ffdfbdd1447b => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:37.908639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11195c35-9ad9-4ae1-93eb-b369a5d3f5cf None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:37.910285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11195c35-9ad9-4ae1-93eb-b369a5d3f5cf None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:37.910777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11195c35-9ad9-4ae1-93eb-b369a5d3f5cf None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:37.912737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 137/270] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:37.918763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c25a98-2243-4fcd-b907-220ed6e5121c None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:37.919345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c25a98-2243-4fcd-b907-220ed6e5121c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:37.919821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c25a98-2243-4fcd-b907-220ed6e5121c None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:37.924749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 134/271] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:37.925221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4553d4d-ecee-4eff-91a4-1c82311b881d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:37.925710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4553d4d-ecee-4eff-91a4-1c82311b881d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:37.925811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f9162-8c61-4a0c-831e-5667b8dc116c None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:37.926778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f9162-8c61-4a0c-831e-5667b8dc116c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:37.926778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f9162-8c61-4a0c-831e-5667b8dc116c None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:37.928192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 135/272] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:37.928401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4553d4d-ecee-4eff-91a4-1c82311b881d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:37.934145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759944-b078-4370-af07-bf9e7e5ea9ba None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:37.935693 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759944-b078-4370-af07-bf9e7e5ea9ba None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:37.935693 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7759944-b078-4370-af07-bf9e7e5ea9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:37.951510 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f4553d4d-ecee-4eff-91a4-1c82311b881d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:37.953353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f4553d4d-ecee-4eff-91a4-1c82311b881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:37.953578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d7759944-b078-4370-af07-bf9e7e5ea9ba None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:37.954770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d7759944-b078-4370-af07-bf9e7e5ea9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:38.042077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 136/273] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:38.043729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5faecc-f68c-47aa-bd5f-972c6c8d3b51 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.044167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5faecc-f68c-47aa-bd5f-972c6c8d3b51 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.044477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5faecc-f68c-47aa-bd5f-972c6c8d3b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.050645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 138/274] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:38.051983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631ddb0-6f55-4c67-b0b3-421786d59545 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.052276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631ddb0-6f55-4c67-b0b3-421786d59545 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.052580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631ddb0-6f55-4c67-b0b3-421786d59545 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.053855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 139/275] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:38.059649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204418b2-eb61-42a9-ac7c-2e813d62175c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.059932 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204418b2-eb61-42a9-ac7c-2e813d62175c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.060207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204418b2-eb61-42a9-ac7c-2e813d62175c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.070502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-aa5faecc-f68c-47aa-bd5f-972c6c8d3b51 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:38.071690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aa5faecc-f68c-47aa-bd5f-972c6c8d3b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:38.081189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-204418b2-eb61-42a9-ac7c-2e813d62175c None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:38.082353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-204418b2-eb61-42a9-ac7c-2e813d62175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:38.091557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:38.138829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 137/276] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:38.139808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5faecc-f68c-47aa-bd5f-972c6c8d3b51 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.160616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 140/277] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:38.160616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204418b2-eb61-42a9-ac7c-2e813d62175c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.185995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:38.243157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5faecc-f68c-47aa-bd5f-972c6c8d3b51 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:38.265275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204418b2-eb61-42a9-ac7c-2e813d62175c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:38.277651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea695-def3-47bd-b273-195bbe6dc1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:38.278448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea695-def3-47bd-b273-195bbe6dc1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.278739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea695-def3-47bd-b273-195bbe6dc1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.279020 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea695-def3-47bd-b273-195bbe6dc1b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.289099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ea695-def3-47bd-b273-195bbe6dc1b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e06ea695-def3-47bd-b273-195bbe6dc1b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:38.297412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:38.298376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:38.299174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:38.299768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b6b7b-6533-4e44-b37c-883812eb2def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:38.299863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 138/278] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:38.299931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b6b7b-6533-4e44-b37c-883812eb2def None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.300004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea695-def3-47bd-b273-195bbe6dc1b3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.300071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b6b7b-6533-4e44-b37c-883812eb2def None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.300071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b6b7b-6533-4e44-b37c-883812eb2def None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.315854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b6b7b-6533-4e44-b37c-883812eb2def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-386b6b7b-6533-4e44-b37c-883812eb2def None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:38.323724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:38.324972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:38.325767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 141/279] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:38.326420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b6b7b-6533-4e44-b37c-883812eb2def None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.330087 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:38.400380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:38.403599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea695-def3-47bd-b273-195bbe6dc1b3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:38.426215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b6b7b-6533-4e44-b37c-883812eb2def None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:38.439303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e44586-d705-4345-a7da-4901bc220150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:38.440092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e44586-d705-4345-a7da-4901bc220150 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.440410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e44586-d705-4345-a7da-4901bc220150 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.440692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e44586-d705-4345-a7da-4901bc220150 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.441843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e44586-d705-4345-a7da-4901bc220150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:38.443289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e44586-d705-4345-a7da-4901bc220150 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:38.445024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 139/280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:38.445783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e44586-d705-4345-a7da-4901bc220150 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.465602 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8aaa9b-080d-47d8-9549-beba0bbc2d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:38.466394 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8aaa9b-080d-47d8-9549-beba0bbc2d14 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.466676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8aaa9b-080d-47d8-9549-beba0bbc2d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.466951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8aaa9b-080d-47d8-9549-beba0bbc2d14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.477331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8aaa9b-080d-47d8-9549-beba0bbc2d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-1a8aaa9b-080d-47d8-9549-beba0bbc2d14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:38.485553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:38.486423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:38.487203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 142/281] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:38.487836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8aaa9b-080d-47d8-9549-beba0bbc2d14 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.548136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e44586-d705-4345-a7da-4901bc220150 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:38.548438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:38.587664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8aaa9b-080d-47d8-9549-beba0bbc2d14 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:38.587878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ac4ba-5864-4278-9ec1-759dc183fe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:38.588570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ac4ba-5864-4278-9ec1-759dc183fe68 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.588854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ac4ba-5864-4278-9ec1-759dc183fe68 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.589725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ac4ba-5864-4278-9ec1-759dc183fe68 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.590474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ac4ba-5864-4278-9ec1-759dc183fe68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:38.590778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ac4ba-5864-4278-9ec1-759dc183fe68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:38.592165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ac4ba-5864-4278-9ec1-759dc183fe68 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:38.601028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 140/282] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:38.601816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ac4ba-5864-4278-9ec1-759dc183fe68 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.627533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e415c-2416-4266-902c-39f100f6fed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:38.628202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e415c-2416-4266-902c-39f100f6fed0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.628523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e415c-2416-4266-902c-39f100f6fed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.628802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e415c-2416-4266-902c-39f100f6fed0 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.629547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e415c-2416-4266-902c-39f100f6fed0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:38.629858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e415c-2416-4266-902c-39f100f6fed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:38.631262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e415c-2416-4266-902c-39f100f6fed0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:38.633746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:38.639651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 143/283] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:38.640497 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e415c-2416-4266-902c-39f100f6fed0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.704677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ac4ba-5864-4278-9ec1-759dc183fe68 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:38.737725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08f467-efe1-485b-9ae2-7e3e1ccc94ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:38.738387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08f467-efe1-485b-9ae2-7e3e1ccc94ec None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.738666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08f467-efe1-485b-9ae2-7e3e1ccc94ec None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.738939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08f467-efe1-485b-9ae2-7e3e1ccc94ec None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.739160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e415c-2416-4266-902c-39f100f6fed0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:38.739632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08f467-efe1-485b-9ae2-7e3e1ccc94ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:38.741097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08f467-efe1-485b-9ae2-7e3e1ccc94ec None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:38.760974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 141/284] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:20:38 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:38.761776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08f467-efe1-485b-9ae2-7e3e1ccc94ec None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.780141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:38.790748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370abc5f-f529-4c8b-8c39-523de4e5de00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:38.791608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370abc5f-f529-4c8b-8c39-523de4e5de00 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.792042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370abc5f-f529-4c8b-8c39-523de4e5de00 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.793042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370abc5f-f529-4c8b-8c39-523de4e5de00 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.793832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370abc5f-f529-4c8b-8c39-523de4e5de00 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:38.794149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370abc5f-f529-4c8b-8c39-523de4e5de00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:38.795738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370abc5f-f529-4c8b-8c39-523de4e5de00 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:38.810827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 144/285] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:38.811672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370abc5f-f529-4c8b-8c39-523de4e5de00 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.856664 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:38.877245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08f467-efe1-485b-9ae2-7e3e1ccc94ec None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:38.924770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03a8c9-1ee5-45e6-97b2-ed44e0201951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:38.926551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03a8c9-1ee5-45e6-97b2-ed44e0201951 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.926916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03a8c9-1ee5-45e6-97b2-ed44e0201951 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.927257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03a8c9-1ee5-45e6-97b2-ed44e0201951 None admin] PATH_INFO: `/v3/users/ee7220c334af420291c478e2dd299667` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.928543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03a8c9-1ee5-45e6-97b2-ed44e0201951 None admin] RBAC: Authorizing `identity:get_user(user_id=ee7220c334af420291c478e2dd299667)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:38.931325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03a8c9-1ee5-45e6-97b2-ed44e0201951 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:38.936328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 142/286] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:38 2022] GET /identity/v3/users/ee7220c334af420291c478e2dd299667 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:38.936328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03a8c9-1ee5-45e6-97b2-ed44e0201951 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:38.937543 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370abc5f-f529-4c8b-8c39-523de4e5de00 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:38.980127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6864d-072e-4d8d-9168-faf70651dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:38.981748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6864d-072e-4d8d-9168-faf70651dd65 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:38.982037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6864d-072e-4d8d-9168-faf70651dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:38.982318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6864d-072e-4d8d-9168-faf70651dd65 None admin] PATH_INFO: `/v3/users/b8e1e872ffce467384b72503829c9b21` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:38.998079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6864d-072e-4d8d-9168-faf70651dd65 None admin] RBAC: Authorizing `identity:get_user(user_id=b8e1e872ffce467384b72503829c9b21)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:38.999566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6864d-072e-4d8d-9168-faf70651dd65 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:39.002716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 145/287] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:38 2022] GET /identity/v3/users/b8e1e872ffce467384b72503829c9b21 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:39.003534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6864d-072e-4d8d-9168-faf70651dd65 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:39.055096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03a8c9-1ee5-45e6-97b2-ed44e0201951 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:39.062233 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:39.103751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82c8e6-6f83-44af-a2a0-15ec9c2dac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:39.104773 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82c8e6-6f83-44af-a2a0-15ec9c2dac4f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:39.105134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82c8e6-6f83-44af-a2a0-15ec9c2dac4f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:39.105498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82c8e6-6f83-44af-a2a0-15ec9c2dac4f None admin] PATH_INFO: `/v3/users/23b0c52acfdb4d4a81b1be970cda509b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:39.106720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82c8e6-6f83-44af-a2a0-15ec9c2dac4f None admin] RBAC: Authorizing `identity:get_user(user_id=23b0c52acfdb4d4a81b1be970cda509b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:39.108170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82c8e6-6f83-44af-a2a0-15ec9c2dac4f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:39.110002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 143/288] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:38 2022] GET /identity/v3/users/23b0c52acfdb4d4a81b1be970cda509b => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:39.112823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82c8e6-6f83-44af-a2a0-15ec9c2dac4f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:39.127963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6864d-072e-4d8d-9168-faf70651dd65 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:39.144121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:39.168649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215c2d-e249-41d6-b61c-c13c753c218f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:39.169695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215c2d-e249-41d6-b61c-c13c753c218f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:39.170304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215c2d-e249-41d6-b61c-c13c753c218f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:39.170304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215c2d-e249-41d6-b61c-c13c753c218f None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:39.171591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88215c2d-e249-41d6-b61c-c13c753c218f None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:39.173105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88215c2d-e249-41d6-b61c-c13c753c218f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:39.175160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 146/289] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:39 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:39.175803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215c2d-e249-41d6-b61c-c13c753c218f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:39.227845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82c8e6-6f83-44af-a2a0-15ec9c2dac4f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:39.267788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdcc5b-aefd-4449-80fc-bd99bc076870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:39.268766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdcc5b-aefd-4449-80fc-bd99bc076870 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:39.269225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdcc5b-aefd-4449-80fc-bd99bc076870 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:39.269704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdcc5b-aefd-4449-80fc-bd99bc076870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:39.270805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdcc5b-aefd-4449-80fc-bd99bc076870 None admin] RBAC: Adding query filter params (role.id=d2c2845de1d548bb9cfb46f70619b3f8, user.id=b8e1e872ffce467384b72503829c9b21, scope.system=all) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:39.271316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdcc5b-aefd-4449-80fc-bd99bc076870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:39.273009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdcc5b-aefd-4449-80fc-bd99bc076870 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:39.278810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 144/290] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Jul 21 17:20:39 2022] GET /identity/v3/role_assignments?user.id=b8e1e872ffce467384b72503829c9b21&scope.system=all&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:39.285995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdcc5b-aefd-4449-80fc-bd99bc076870 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:39.293408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215c2d-e249-41d6-b61c-c13c753c218f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:39.312203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:39.340778 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc0bdc-63a0-43fb-a113-b972bcb7bf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:39.341444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc0bdc-63a0-43fb-a113-b972bcb7bf85 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:39.341722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc0bdc-63a0-43fb-a113-b972bcb7bf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:39.341994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc0bdc-63a0-43fb-a113-b972bcb7bf85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:39.429446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc0bdc-63a0-43fb-a113-b972bcb7bf85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:39.431195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc0bdc-63a0-43fb-a113-b972bcb7bf85 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:39.512390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdcc5b-aefd-4449-80fc-bd99bc076870 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:39.528740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:39.552982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce3271-e893-48a1-a367-e7d318c56acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:39.553855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce3271-e893-48a1-a367-e7d318c56acf None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:39.554145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce3271-e893-48a1-a367-e7d318c56acf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:39.554425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce3271-e893-48a1-a367-e7d318c56acf None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/users/ee7220c334af420291c478e2dd299667/roles/feffc80bdc3641a59c6d5d545c7fce94` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:39.556483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce3271-e893-48a1-a367-e7d318c56acf None admin] RBAC: Authorizing `identity:create_grant(project_id=4819e5a17eb24efcb8ccbb51a48b30ca, user_id=ee7220c334af420291c478e2dd299667, role_id=feffc80bdc3641a59c6d5d545c7fce94)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:39.559070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce3271-e893-48a1-a367-e7d318c56acf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:39.576175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 145/291] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:20:39 2022] PUT /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/users/ee7220c334af420291c478e2dd299667/roles/feffc80bdc3641a59c6d5d545c7fce94 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:39.613501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 147/292] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:39.619412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce3271-e893-48a1-a367-e7d318c56acf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:39.755107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce3271-e893-48a1-a367-e7d318c56acf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:39.795542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc33ec-79d3-4626-9c2c-4f9490d57942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:39.796422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc33ec-79d3-4626-9c2c-4f9490d57942 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:39.796706 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc33ec-79d3-4626-9c2c-4f9490d57942 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:39.796977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc33ec-79d3-4626-9c2c-4f9490d57942 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:39.798164 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc33ec-79d3-4626-9c2c-4f9490d57942 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:39.799630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc33ec-79d3-4626-9c2c-4f9490d57942 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:39.801522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 146/293] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:39 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:39.807338 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc0bdc-63a0-43fb-a113-b972bcb7bf85 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:39.825402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:39.921933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc0bdc-63a0-43fb-a113-b972bcb7bf85 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:39.961799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fff56-5256-4e43-8a19-1447cf84a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:39.963270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fff56-5256-4e43-8a19-1447cf84a943 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:39.963552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fff56-5256-4e43-8a19-1447cf84a943 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:39.963822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fff56-5256-4e43-8a19-1447cf84a943 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:39.965694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fff56-5256-4e43-8a19-1447cf84a943 None admin] RBAC: Adding query filter params (role.id=d2c2845de1d548bb9cfb46f70619b3f8, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca, user.id=23b0c52acfdb4d4a81b1be970cda509b) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:39.966093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fff56-5256-4e43-8a19-1447cf84a943 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:39.967772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fff56-5256-4e43-8a19-1447cf84a943 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:39.974531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 148/294] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:39 2022] GET /identity/v3/role_assignments?user.id=23b0c52acfdb4d4a81b1be970cda509b&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:41.133467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1129ec-34f7-413f-8d26-3b66e6d55454 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.134001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1129ec-34f7-413f-8d26-3b66e6d55454 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.134001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1129ec-34f7-413f-8d26-3b66e6d55454 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.135731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 147/295] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:41.141804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce510f4c-48ce-4123-bcd6-0ca9ef4be418 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.142231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce510f4c-48ce-4123-bcd6-0ca9ef4be418 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.142428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce510f4c-48ce-4123-bcd6-0ca9ef4be418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.158886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ce510f4c-48ce-4123-bcd6-0ca9ef4be418 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:41.160058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ce510f4c-48ce-4123-bcd6-0ca9ef4be418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:41.173353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6b2ab-3702-405b-8d38-76a691ce648d None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.173924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6b2ab-3702-405b-8d38-76a691ce648d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.174446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6b2ab-3702-405b-8d38-76a691ce648d None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.176278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 148/296] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:41.181000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac5cce-cdd1-4c8f-a829-fb8d1b0459c9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.181289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac5cce-cdd1-4c8f-a829-fb8d1b0459c9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.181565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac5cce-cdd1-4c8f-a829-fb8d1b0459c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.205625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a6ac5cce-cdd1-4c8f-a829-fb8d1b0459c9 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:41.207351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a6ac5cce-cdd1-4c8f-a829-fb8d1b0459c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:41.221978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:41.252411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 149/297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:41.271647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce510f4c-48ce-4123-bcd6-0ca9ef4be418 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:41.284550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:41.308378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 149/298] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:41.324015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac5cce-cdd1-4c8f-a829-fb8d1b0459c9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:41.389314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce510f4c-48ce-4123-bcd6-0ca9ef4be418 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:41.424024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fb2b2-e244-4af3-b04e-97bcb9533936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:41.425188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fb2b2-e244-4af3-b04e-97bcb9533936 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.425689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fb2b2-e244-4af3-b04e-97bcb9533936 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.426153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fb2b2-e244-4af3-b04e-97bcb9533936 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.437749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fb2b2-e244-4af3-b04e-97bcb9533936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:41.443057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac5cce-cdd1-4c8f-a829-fb8d1b0459c9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6a1fb2b2-e244-4af3-b04e-97bcb9533936 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:41.446168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:41.447037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:41.447814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:41.456408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 150/299] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:41.458459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fb2b2-e244-4af3-b04e-97bcb9533936 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:41.481637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590b81d-b635-4c31-ae57-217c49f2b334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:41.482481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590b81d-b635-4c31-ae57-217c49f2b334 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.482718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590b81d-b635-4c31-ae57-217c49f2b334 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.482993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590b81d-b635-4c31-ae57-217c49f2b334 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.484167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590b81d-b635-4c31-ae57-217c49f2b334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:41.485866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590b81d-b635-4c31-ae57-217c49f2b334 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:41.492082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 150/300] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:41.493340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590b81d-b635-4c31-ae57-217c49f2b334 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:41.513504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:41.552923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:41.587943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fb2b2-e244-4af3-b04e-97bcb9533936 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:41.611757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590b81d-b635-4c31-ae57-217c49f2b334 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:41.626230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183a548-58de-4c4e-8ef0-f04a2680c23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:41.626966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183a548-58de-4c4e-8ef0-f04a2680c23a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.627244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183a548-58de-4c4e-8ef0-f04a2680c23a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.627518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183a548-58de-4c4e-8ef0-f04a2680c23a None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.628406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183a548-58de-4c4e-8ef0-f04a2680c23a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:41.628830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183a548-58de-4c4e-8ef0-f04a2680c23a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:41.630372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183a548-58de-4c4e-8ef0-f04a2680c23a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:41.640998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 151/301] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:41.642288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea39eea-97d8-4e68-884b-46a9c939be93 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.642573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea39eea-97d8-4e68-884b-46a9c939be93 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.642853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea39eea-97d8-4e68-884b-46a9c939be93 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.644310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 152/302] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:41.648817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea39eea-97d8-4e68-884b-46a9c939be93 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:41.652726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f16212-f197-4e3e-a654-41b1bffe3905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:41.653312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f16212-f197-4e3e-a654-41b1bffe3905 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.653593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f16212-f197-4e3e-a654-41b1bffe3905 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.653866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f16212-f197-4e3e-a654-41b1bffe3905 None admin] PATH_INFO: `/v3/groups` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.654564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f16212-f197-4e3e-a654-41b1bffe3905 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:41.655991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f16212-f197-4e3e-a654-41b1bffe3905 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:41.673791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 151/303] 10.222.0.57 () {60 vars in 1134 bytes} [Thu Jul 21 17:20:41 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:41.677688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeddc35-50d7-4786-bb0c-fa5916de76c2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.678215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeddc35-50d7-4786-bb0c-fa5916de76c2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.678668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeddc35-50d7-4786-bb0c-fa5916de76c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.704209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cfeddc35-50d7-4786-bb0c-fa5916de76c2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:41.705744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cfeddc35-50d7-4786-bb0c-fa5916de76c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:41.793758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 152/304] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:41.805611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea39eea-97d8-4e68-884b-46a9c939be93 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:41.813390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeddc35-50d7-4786-bb0c-fa5916de76c2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:41.816802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:41.843780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:41.853357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9aea33-5144-470b-a46c-4efd631d2511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:41.854235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9aea33-5144-470b-a46c-4efd631d2511 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:41.854518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9aea33-5144-470b-a46c-4efd631d2511 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:41.855538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9aea33-5144-470b-a46c-4efd631d2511 None admin] PATH_INFO: `/v3/users/5f6f35c851064dfd8151818af98136f9` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:41.877250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9aea33-5144-470b-a46c-4efd631d2511 None admin] RBAC: Authorizing `identity:get_user(user_id=5f6f35c851064dfd8151818af98136f9)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:41.877870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9aea33-5144-470b-a46c-4efd631d2511 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:41.879911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 153/305] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:41 2022] GET /identity/v3/users/5f6f35c851064dfd8151818af98136f9 => generated 293 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:41.886423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9aea33-5144-470b-a46c-4efd631d2511 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:41.956365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeddc35-50d7-4786-bb0c-fa5916de76c2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:42.004575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635ea36-6ae4-40c2-ba0c-b190559112dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:42.005911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635ea36-6ae4-40c2-ba0c-b190559112dc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.006308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635ea36-6ae4-40c2-ba0c-b190559112dc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.006588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635ea36-6ae4-40c2-ba0c-b190559112dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.008162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9aea33-5144-470b-a46c-4efd631d2511 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:42.019659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635ea36-6ae4-40c2-ba0c-b190559112dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-d635ea36-6ae4-40c2-ba0c-b190559112dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:42.028878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:42.029788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:42.031331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 153/306] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:42.032051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658e957-aa8d-4a51-8e29-439f939eed1b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.032051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658e957-aa8d-4a51-8e29-439f939eed1b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.032051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658e957-aa8d-4a51-8e29-439f939eed1b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.033485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 154/307] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:42.037875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658e957-aa8d-4a51-8e29-439f939eed1b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:42.055492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f7456-3196-4ef4-a093-b5234780927e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:42.056257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f7456-3196-4ef4-a093-b5234780927e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.056583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f7456-3196-4ef4-a093-b5234780927e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.057008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f7456-3196-4ef4-a093-b5234780927e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.058997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f7456-3196-4ef4-a093-b5234780927e None admin] RBAC: Adding query filter params (role.id=feffc80bdc3641a59c6d5d545c7fce94, user.id=5f6f35c851064dfd8151818af98136f9, scope.system=all) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:42.059288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f7456-3196-4ef4-a093-b5234780927e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:42.060817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f7456-3196-4ef4-a093-b5234780927e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:42.067203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 154/308] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Jul 21 17:20:41 2022] GET /identity/v3/role_assignments?user.id=5f6f35c851064dfd8151818af98136f9&scope.system=all&role.id=feffc80bdc3641a59c6d5d545c7fce94 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:42.068489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5c1df-de26-4117-9e46-4c0511f1d631 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.068775 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5c1df-de26-4117-9e46-4c0511f1d631 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.069051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5c1df-de26-4117-9e46-4c0511f1d631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.087087 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-47c5c1df-de26-4117-9e46-4c0511f1d631 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:42.088430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-47c5c1df-de26-4117-9e46-4c0511f1d631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:42.122384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:42.146794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:42.175403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 155/309] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:42.185362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658e957-aa8d-4a51-8e29-439f939eed1b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:42.199888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5c1df-de26-4117-9e46-4c0511f1d631 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:42.230931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3facf-1707-4dd1-9b96-2cae690135ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:42.231871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3facf-1707-4dd1-9b96-2cae690135ab None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.232282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3facf-1707-4dd1-9b96-2cae690135ab None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.232702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3facf-1707-4dd1-9b96-2cae690135ab None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.233610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3facf-1707-4dd1-9b96-2cae690135ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:42.240347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3facf-1707-4dd1-9b96-2cae690135ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:42.240347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3facf-1707-4dd1-9b96-2cae690135ab None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:42.244683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 155/310] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:42.250918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3facf-1707-4dd1-9b96-2cae690135ab None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:42.326757 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5c1df-de26-4117-9e46-4c0511f1d631 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:42.357957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3facf-1707-4dd1-9b96-2cae690135ab None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:42.364624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933ce4-3391-4c1f-93ab-8b11b611c8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:42.365626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933ce4-3391-4c1f-93ab-8b11b611c8ac None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.366057 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933ce4-3391-4c1f-93ab-8b11b611c8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.366475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933ce4-3391-4c1f-93ab-8b11b611c8ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.378444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79933ce4-3391-4c1f-93ab-8b11b611c8ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:42.382821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-79933ce4-3391-4c1f-93ab-8b11b611c8ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:42.387858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:42.388986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:42.389848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:42.396200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09943cbd-ea92-41c8-861d-8a4fd95ef201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:42.396380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09943cbd-ea92-41c8-861d-8a4fd95ef201 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.396789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09943cbd-ea92-41c8-861d-8a4fd95ef201 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.396907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09943cbd-ea92-41c8-861d-8a4fd95ef201 None admin] PATH_INFO: `/v3/users/b8e1e872ffce467384b72503829c9b21` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.397034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 156/311] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:42.398025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933ce4-3391-4c1f-93ab-8b11b611c8ac None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:42.398136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09943cbd-ea92-41c8-861d-8a4fd95ef201 None admin] RBAC: Authorizing `identity:get_user(user_id=b8e1e872ffce467384b72503829c9b21)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:42.399556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09943cbd-ea92-41c8-861d-8a4fd95ef201 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:42.406644 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 156/312] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:42 2022] GET /identity/v3/users/b8e1e872ffce467384b72503829c9b21 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:42.407494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09943cbd-ea92-41c8-861d-8a4fd95ef201 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:42.447428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:42.510862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933ce4-3391-4c1f-93ab-8b11b611c8ac None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:42.519611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09943cbd-ea92-41c8-861d-8a4fd95ef201 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:42.550426 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfe297-ae09-46f9-8cef-9899883c6431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:42.551430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfe297-ae09-46f9-8cef-9899883c6431 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.551859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfe297-ae09-46f9-8cef-9899883c6431 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.552276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfe297-ae09-46f9-8cef-9899883c6431 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.553188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfe297-ae09-46f9-8cef-9899883c6431 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:42.553660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfe297-ae09-46f9-8cef-9899883c6431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:42.555173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfe297-ae09-46f9-8cef-9899883c6431 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:42.555783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47190e08-eaf7-4e79-adb0-16ffd63135a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:42.556781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47190e08-eaf7-4e79-adb0-16ffd63135a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.557051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47190e08-eaf7-4e79-adb0-16ffd63135a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.557328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47190e08-eaf7-4e79-adb0-16ffd63135a1 None admin] PATH_INFO: `/v3/system/users/b8e1e872ffce467384b72503829c9b21/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.559018 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47190e08-eaf7-4e79-adb0-16ffd63135a1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b8e1e872ffce467384b72503829c9b21, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:42.560488 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47190e08-eaf7-4e79-adb0-16ffd63135a1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:42.566000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 157/313] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:42.572213 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfe297-ae09-46f9-8cef-9899883c6431 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:42.572918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 157/314] 10.222.0.57 () {58 vars in 1334 bytes} [Thu Jul 21 17:20:42 2022] PUT /identity/v3/system/users/b8e1e872ffce467384b72503829c9b21/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:42.697751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfe297-ae09-46f9-8cef-9899883c6431 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:42.702963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:42.739247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0024eb-b51f-4239-a791-2e367e3d064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:42.740126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0024eb-b51f-4239-a791-2e367e3d064e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.740511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0024eb-b51f-4239-a791-2e367e3d064e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.740794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0024eb-b51f-4239-a791-2e367e3d064e None admin] PATH_INFO: `/v3/users/ee7220c334af420291c478e2dd299667` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.742007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0024eb-b51f-4239-a791-2e367e3d064e None admin] RBAC: Authorizing `identity:get_user(user_id=ee7220c334af420291c478e2dd299667)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:42.743483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0024eb-b51f-4239-a791-2e367e3d064e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:42.745386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 158/315] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:42 2022] GET /identity/v3/users/ee7220c334af420291c478e2dd299667 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:42.751322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47190e08-eaf7-4e79-adb0-16ffd63135a1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:42.825133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:42.875236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47190e08-eaf7-4e79-adb0-16ffd63135a1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:42.914062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee37078-32a8-4dec-b2d0-497967fdce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:42.915061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee37078-32a8-4dec-b2d0-497967fdce4e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:42.915536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee37078-32a8-4dec-b2d0-497967fdce4e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:42.915989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee37078-32a8-4dec-b2d0-497967fdce4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:42.991763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee37078-32a8-4dec-b2d0-497967fdce4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:42.993161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee37078-32a8-4dec-b2d0-497967fdce4e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:43.052709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:43.100651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 158/316] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:43.107719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0024eb-b51f-4239-a791-2e367e3d064e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:43.219162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0024eb-b51f-4239-a791-2e367e3d064e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:43.254448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3469406-934a-46d3-b579-6e223b48bdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:43.255227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3469406-934a-46d3-b579-6e223b48bdf0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:43.255505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3469406-934a-46d3-b579-6e223b48bdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:43.255782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3469406-934a-46d3-b579-6e223b48bdf0 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:43.256959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3469406-934a-46d3-b579-6e223b48bdf0 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:43.258422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3469406-934a-46d3-b579-6e223b48bdf0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:43.260456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 159/317] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:43 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:43.267084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee37078-32a8-4dec-b2d0-497967fdce4e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:43.371912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee37078-32a8-4dec-b2d0-497967fdce4e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:43.407176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae753a8b-d5a7-4ba7-94e6-ece0a326aca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:43.407843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae753a8b-d5a7-4ba7-94e6-ece0a326aca2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:43.408122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae753a8b-d5a7-4ba7-94e6-ece0a326aca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:43.408426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae753a8b-d5a7-4ba7-94e6-ece0a326aca2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:43.409264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae753a8b-d5a7-4ba7-94e6-ece0a326aca2 None admin] RBAC: Adding query filter params (role.id=feffc80bdc3641a59c6d5d545c7fce94, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca, user.id=ee7220c334af420291c478e2dd299667) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:43.409573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae753a8b-d5a7-4ba7-94e6-ece0a326aca2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:43.411014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae753a8b-d5a7-4ba7-94e6-ece0a326aca2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:43.417969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 159/318] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:20:43 2022] GET /identity/v3/role_assignments?user.id=ee7220c334af420291c478e2dd299667&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=feffc80bdc3641a59c6d5d545c7fce94 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:43.822947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32708deb-bb43-4a52-887f-4e9228b87b3a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:43.823917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32708deb-bb43-4a52-887f-4e9228b87b3a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:43.824257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32708deb-bb43-4a52-887f-4e9228b87b3a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:43.826064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 160/319] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:43.831209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6ea6d-5d10-42b9-898a-2562312409f7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:43.831506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6ea6d-5d10-42b9-898a-2562312409f7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:43.831787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6ea6d-5d10-42b9-898a-2562312409f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:43.849167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-64e6ea6d-5d10-42b9-898a-2562312409f7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:43.850347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-64e6ea6d-5d10-42b9-898a-2562312409f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:43.865150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:43.924812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 160/320] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:43.947815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32708deb-bb43-4a52-887f-4e9228b87b3a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:43.993396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:44.052506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32708deb-bb43-4a52-887f-4e9228b87b3a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:44.086960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ba9c8-f511-4237-9f25-44fe2a962cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:44.087673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba9c8-f511-4237-9f25-44fe2a962cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:44.087957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba9c8-f511-4237-9f25-44fe2a962cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:44.088232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba9c8-f511-4237-9f25-44fe2a962cf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:44.089548 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ba9c8-f511-4237-9f25-44fe2a962cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:44.090983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ba9c8-f511-4237-9f25-44fe2a962cf7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:44.092796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 161/321] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:20:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:44.098258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6ea6d-5d10-42b9-898a-2562312409f7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:44.116883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2aa5f-274d-45c9-b13f-04759e3191e5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:44.117212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2aa5f-274d-45c9-b13f-04759e3191e5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:44.117440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2aa5f-274d-45c9-b13f-04759e3191e5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:44.118917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 162/322] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:44.124058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95bf1-d775-454b-998a-7155af921b06 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:44.124374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95bf1-d775-454b-998a-7155af921b06 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:44.124654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95bf1-d775-454b-998a-7155af921b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:44.141519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d8d95bf1-d775-454b-998a-7155af921b06 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:44.142698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d8d95bf1-d775-454b-998a-7155af921b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:44.208006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6ea6d-5d10-42b9-898a-2562312409f7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:44.220627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 163/323] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:44.242780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d95bf1-d775-454b-998a-7155af921b06 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:44.243748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e06932-0d2a-40f7-abcc-0a1fc44250cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:44.244469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e06932-0d2a-40f7-abcc-0a1fc44250cf None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:44.244932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e06932-0d2a-40f7-abcc-0a1fc44250cf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:44.245763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e06932-0d2a-40f7-abcc-0a1fc44250cf None admin] PATH_INFO: `/v3/groups` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:44.247371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e06932-0d2a-40f7-abcc-0a1fc44250cf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:44.248867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e06932-0d2a-40f7-abcc-0a1fc44250cf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:44.266534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 161/324] 10.222.0.57 () {60 vars in 1134 bytes} [Thu Jul 21 17:20:44 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:20:44.289698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:44.360216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d95bf1-d775-454b-998a-7155af921b06 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:44.397381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622253da-7dbd-4ea9-9f91-891ace5953bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:44.398194 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622253da-7dbd-4ea9-9f91-891ace5953bc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:44.398476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622253da-7dbd-4ea9-9f91-891ace5953bc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:44.398748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622253da-7dbd-4ea9-9f91-891ace5953bc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:44.410448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622253da-7dbd-4ea9-9f91-891ace5953bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-622253da-7dbd-4ea9-9f91-891ace5953bc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:44.418979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:44.419879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:44.420684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 164/325] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:44.425935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e06932-0d2a-40f7-abcc-0a1fc44250cf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:44.440239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:44.532641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e06932-0d2a-40f7-abcc-0a1fc44250cf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:44.568638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0f5ac-e2c6-40f3-b861-a2c1ab2cfd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:44.569315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0f5ac-e2c6-40f3-b861-a2c1ab2cfd19 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:44.569598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0f5ac-e2c6-40f3-b861-a2c1ab2cfd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:44.569871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0f5ac-e2c6-40f3-b861-a2c1ab2cfd19 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:44.570680 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0f5ac-e2c6-40f3-b861-a2c1ab2cfd19 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:44.570992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0f5ac-e2c6-40f3-b861-a2c1ab2cfd19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:44.572383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0f5ac-e2c6-40f3-b861-a2c1ab2cfd19 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:44.581654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 162/326] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:44.587031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622253da-7dbd-4ea9-9f91-891ace5953bc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:44.691777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:44.697107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622253da-7dbd-4ea9-9f91-891ace5953bc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:44.735120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224f320-bee4-470c-9d43-3fe9c0844183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:44.735995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f320-bee4-470c-9d43-3fe9c0844183 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:44.736272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f320-bee4-470c-9d43-3fe9c0844183 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:44.736576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f320-bee4-470c-9d43-3fe9c0844183 None admin] PATH_INFO: `/v3/users/5f6f35c851064dfd8151818af98136f9` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:44.737774 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224f320-bee4-470c-9d43-3fe9c0844183 None admin] RBAC: Authorizing `identity:get_user(user_id=5f6f35c851064dfd8151818af98136f9)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:44.739237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224f320-bee4-470c-9d43-3fe9c0844183 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:44.740907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 165/327] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:44 2022] GET /identity/v3/users/5f6f35c851064dfd8151818af98136f9 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:44.746755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0f5ac-e2c6-40f3-b861-a2c1ab2cfd19 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:44.791085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faad483-a6bd-4054-a21c-cc282466f51e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:44.791085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faad483-a6bd-4054-a21c-cc282466f51e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:44.791085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faad483-a6bd-4054-a21c-cc282466f51e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:44.792767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 166/328] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:44.798377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a479b8e-b5ec-4817-8503-82d80bbf6c8a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:44.798669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a479b8e-b5ec-4817-8503-82d80bbf6c8a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:44.798945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a479b8e-b5ec-4817-8503-82d80bbf6c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:44.818110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5a479b8e-b5ec-4817-8503-82d80bbf6c8a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:44.819675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5a479b8e-b5ec-4817-8503-82d80bbf6c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:44.828452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:44.862673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0f5ac-e2c6-40f3-b861-a2c1ab2cfd19 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:44.905664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 167/329] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:44.919423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0563d0-2859-46a1-a9d4-ce0f00ca13db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:44.919613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0563d0-2859-46a1-a9d4-ce0f00ca13db None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:44.919885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0563d0-2859-46a1-a9d4-ce0f00ca13db None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:44.920165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0563d0-2859-46a1-a9d4-ce0f00ca13db None admin] PATH_INFO: `/v3/system/users/5f6f35c851064dfd8151818af98136f9/roles/feffc80bdc3641a59c6d5d545c7fce94` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:44.925053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0563d0-2859-46a1-a9d4-ce0f00ca13db None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5f6f35c851064dfd8151818af98136f9, role_id=feffc80bdc3641a59c6d5d545c7fce94)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:44.926446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0563d0-2859-46a1-a9d4-ce0f00ca13db None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:44.936920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a479b8e-b5ec-4817-8503-82d80bbf6c8a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:44.939982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 163/330] 10.222.0.57 () {58 vars in 1334 bytes} [Thu Jul 21 17:20:44 2022] PUT /identity/v3/system/users/5f6f35c851064dfd8151818af98136f9/roles/feffc80bdc3641a59c6d5d545c7fce94 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:45.056800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:45.064589 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a479b8e-b5ec-4817-8503-82d80bbf6c8a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:45.108774 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca636d-467d-4571-837f-728f9cf64635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:45.109603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca636d-467d-4571-837f-728f9cf64635 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:45.109879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca636d-467d-4571-837f-728f9cf64635 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:45.110150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca636d-467d-4571-837f-728f9cf64635 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:45.122678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca636d-467d-4571-837f-728f9cf64635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-10ca636d-467d-4571-837f-728f9cf64635 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:45.132430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:45.133375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:45.134168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 168/331] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:45.142231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0563d0-2859-46a1-a9d4-ce0f00ca13db None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:45.266456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0563d0-2859-46a1-a9d4-ce0f00ca13db None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:45.302365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e47bca-aa21-48ba-b7af-105f888838a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:45.303126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e47bca-aa21-48ba-b7af-105f888838a9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:45.303318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e47bca-aa21-48ba-b7af-105f888838a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:45.303593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e47bca-aa21-48ba-b7af-105f888838a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:45.304355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e47bca-aa21-48ba-b7af-105f888838a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:45.304692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e47bca-aa21-48ba-b7af-105f888838a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:45.306047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e47bca-aa21-48ba-b7af-105f888838a9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:45.308981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:45.322685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 164/332] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:45.329045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca636d-467d-4571-837f-728f9cf64635 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:45.455919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca636d-467d-4571-837f-728f9cf64635 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:45.498595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd174b-3b87-4c8c-81e2-6de1a670b24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:45.499485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd174b-3b87-4c8c-81e2-6de1a670b24c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:45.499811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd174b-3b87-4c8c-81e2-6de1a670b24c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:45.500092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd174b-3b87-4c8c-81e2-6de1a670b24c None admin] PATH_INFO: `/v3/users/b8e1e872ffce467384b72503829c9b21` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:45.501677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd174b-3b87-4c8c-81e2-6de1a670b24c None admin] RBAC: Authorizing `identity:get_user(user_id=b8e1e872ffce467384b72503829c9b21)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:45.503169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd174b-3b87-4c8c-81e2-6de1a670b24c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:45.505033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 169/333] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:45 2022] GET /identity/v3/users/b8e1e872ffce467384b72503829c9b21 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:45.510702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e47bca-aa21-48ba-b7af-105f888838a9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:45.624978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e47bca-aa21-48ba-b7af-105f888838a9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:45.661020 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ed80a-e40d-4689-9efa-73a33bf16a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:45.661775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ed80a-e40d-4689-9efa-73a33bf16a27 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:45.662061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ed80a-e40d-4689-9efa-73a33bf16a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:45.662379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ed80a-e40d-4689-9efa-73a33bf16a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:45.663225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ed80a-e40d-4689-9efa-73a33bf16a27 None admin] RBAC: Adding query filter params (role.id=d2c2845de1d548bb9cfb46f70619b3f8, user.id=b8e1e872ffce467384b72503829c9b21, scope.system=all) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:45.663535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ed80a-e40d-4689-9efa-73a33bf16a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:45.665050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ed80a-e40d-4689-9efa-73a33bf16a27 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:45.678868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 165/334] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Jul 21 17:20:45 2022] GET /identity/v3/role_assignments?user.id=b8e1e872ffce467384b72503829c9b21&scope.system=all&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 510 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:46.812773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0dd88-593f-4acf-af34-59a83c2be218 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:46.813270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0dd88-593f-4acf-af34-59a83c2be218 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:46.813344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0dd88-593f-4acf-af34-59a83c2be218 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:46.815030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 170/335] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:46.821206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2813cab-2768-41c3-bc21-c4504b9e5eb6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:46.821610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2813cab-2768-41c3-bc21-c4504b9e5eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:46.821777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2813cab-2768-41c3-bc21-c4504b9e5eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:46.844969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e2813cab-2768-41c3-bc21-c4504b9e5eb6 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:46.846164 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e2813cab-2768-41c3-bc21-c4504b9e5eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:46.911815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:46.950476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 166/336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:46.993837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0dd88-593f-4acf-af34-59a83c2be218 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:47.025331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:47.118746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0dd88-593f-4acf-af34-59a83c2be218 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:47.164097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fcffb-765e-4875-ba52-cff9b791ebc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:47.164950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fcffb-765e-4875-ba52-cff9b791ebc8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:47.165231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fcffb-765e-4875-ba52-cff9b791ebc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:47.165504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fcffb-765e-4875-ba52-cff9b791ebc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:47.178319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79fcffb-765e-4875-ba52-cff9b791ebc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f79fcffb-765e-4875-ba52-cff9b791ebc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:47.186292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:47.187148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:47.187939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:47.192498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 171/337] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:47.195395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2813cab-2768-41c3-bc21-c4504b9e5eb6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:47.286074 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c34f34-53fd-468c-9c20-e0a2408687d5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:47.286387 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c34f34-53fd-468c-9c20-e0a2408687d5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:47.286618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c34f34-53fd-468c-9c20-e0a2408687d5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:47.287998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 172/338] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:47.293248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8eeb5-e68a-4053-9ed5-1127ec0c3e54 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:47.293533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8eeb5-e68a-4053-9ed5-1127ec0c3e54 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:47.293812 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8eeb5-e68a-4053-9ed5-1127ec0c3e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:47.313226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8ce8eeb5-e68a-4053-9ed5-1127ec0c3e54 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:47.314408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8ce8eeb5-e68a-4053-9ed5-1127ec0c3e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:47.324625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2813cab-2768-41c3-bc21-c4504b9e5eb6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:47.403715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f5a30-1055-4547-895e-21a4e321d462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:47.404795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f5a30-1055-4547-895e-21a4e321d462 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:47.405324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f5a30-1055-4547-895e-21a4e321d462 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:47.405825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f5a30-1055-4547-895e-21a4e321d462 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:47.406769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f5a30-1055-4547-895e-21a4e321d462 None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:47.407277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f5a30-1055-4547-895e-21a4e321d462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:47.408870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f5a30-1055-4547-895e-21a4e321d462 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:47.427696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 167/339] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:47.433729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f5a30-1055-4547-895e-21a4e321d462 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:47.467377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 173/340] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:47.514655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8eeb5-e68a-4053-9ed5-1127ec0c3e54 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:47.537121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:47.586102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:47.651511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f5a30-1055-4547-895e-21a4e321d462 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:47.660033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8eeb5-e68a-4053-9ed5-1127ec0c3e54 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:47.716804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc07b1f-d057-49ab-84bc-ed13fc9c2116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:47.717401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d93ce-9f08-4d75-8c9a-96e0c8491da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:47.717616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07b1f-d057-49ab-84bc-ed13fc9c2116 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:47.717898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07b1f-d057-49ab-84bc-ed13fc9c2116 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:47.718168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07b1f-d057-49ab-84bc-ed13fc9c2116 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:47.718853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d93ce-9f08-4d75-8c9a-96e0c8491da7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:47.719353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d93ce-9f08-4d75-8c9a-96e0c8491da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:47.719812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d93ce-9f08-4d75-8c9a-96e0c8491da7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:47.729262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc07b1f-d057-49ab-84bc-ed13fc9c2116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-8cc07b1f-d057-49ab-84bc-ed13fc9c2116 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:47.737542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:47.738448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:47.739227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 174/341] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:47.744658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc07b1f-d057-49ab-84bc-ed13fc9c2116 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:47.813171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d93ce-9f08-4d75-8c9a-96e0c8491da7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:47.814981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d93ce-9f08-4d75-8c9a-96e0c8491da7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:47.832320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:47.859530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc07b1f-d057-49ab-84bc-ed13fc9c2116 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:47.899825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:47.902168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bace2f-72e3-4d6e-adc1-1b7d3629f2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:47.903996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bace2f-72e3-4d6e-adc1-1b7d3629f2ca None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:47.904280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bace2f-72e3-4d6e-adc1-1b7d3629f2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:47.904575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bace2f-72e3-4d6e-adc1-1b7d3629f2ca None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:47.905329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bace2f-72e3-4d6e-adc1-1b7d3629f2ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:47.905637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bace2f-72e3-4d6e-adc1-1b7d3629f2ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:47.907027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bace2f-72e3-4d6e-adc1-1b7d3629f2ca None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:47.923854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 175/342] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:47.930659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bace2f-72e3-4d6e-adc1-1b7d3629f2ca None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:48.007456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 168/343] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 17:20:48.013443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d93ce-9f08-4d75-8c9a-96e0c8491da7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:48.040564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bace2f-72e3-4d6e-adc1-1b7d3629f2ca None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:48.077925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc3c9a-23ae-4c1c-896b-65b024359b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:48.078849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc3c9a-23ae-4c1c-896b-65b024359b0c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.079272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc3c9a-23ae-4c1c-896b-65b024359b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.079685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc3c9a-23ae-4c1c-896b-65b024359b0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.098912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:48.213653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc3c9a-23ae-4c1c-896b-65b024359b0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:48.215315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc3c9a-23ae-4c1c-896b-65b024359b0c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:48.224241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d93ce-9f08-4d75-8c9a-96e0c8491da7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:48.264959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519184f-bb92-4814-b3cc-4cbce8579094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:48.266074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519184f-bb92-4814-b3cc-4cbce8579094 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.266501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519184f-bb92-4814-b3cc-4cbce8579094 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.267026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519184f-bb92-4814-b3cc-4cbce8579094 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.268490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519184f-bb92-4814-b3cc-4cbce8579094 None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:48.270090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519184f-bb92-4814-b3cc-4cbce8579094 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:48.276864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 169/344] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:48 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:48.278539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519184f-bb92-4814-b3cc-4cbce8579094 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:48.300827 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:48.393724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519184f-bb92-4814-b3cc-4cbce8579094 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:48.431929 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52d344-44e8-4081-bf6b-cf6933cb7b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:48.433101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52d344-44e8-4081-bf6b-cf6933cb7b74 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.433667 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52d344-44e8-4081-bf6b-cf6933cb7b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.434088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52d344-44e8-4081-bf6b-cf6933cb7b74 None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.441845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52d344-44e8-4081-bf6b-cf6933cb7b74 None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:48.443493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52d344-44e8-4081-bf6b-cf6933cb7b74 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:48.445388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 170/345] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:48 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:48.452035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52d344-44e8-4081-bf6b-cf6933cb7b74 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:48.480357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 176/346] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:48.483054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc3c9a-23ae-4c1c-896b-65b024359b0c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:48.555672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:48.564096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:48.589199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52d344-44e8-4081-bf6b-cf6933cb7b74 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:48.637973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbbaed-917a-400f-9014-1e3c242b7ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:48.638891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbbaed-917a-400f-9014-1e3c242b7ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.639343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbbaed-917a-400f-9014-1e3c242b7ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.639763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbbaed-917a-400f-9014-1e3c242b7ff1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.640788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbbaed-917a-400f-9014-1e3c242b7ff1 None admin] RBAC: Adding query filter params (group.id=3794601340474cd89c8a29d6a289089e, role.id=d2c2845de1d548bb9cfb46f70619b3f8, scope.project.id=380cd1f0793045d280c092a275d9666a) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:48.641308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbbaed-917a-400f-9014-1e3c242b7ff1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:48.643041 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbbaed-917a-400f-9014-1e3c242b7ff1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:48.647706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc3c9a-23ae-4c1c-896b-65b024359b0c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:48.654101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 171/347] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jul 21 17:20:48 2022] GET /identity/v3/role_assignments?group.id=3794601340474cd89c8a29d6a289089e&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:48.657283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b158978-4f47-4ff8-9a1c-48873f654a25 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.657738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b158978-4f47-4ff8-9a1c-48873f654a25 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.658160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b158978-4f47-4ff8-9a1c-48873f654a25 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.659708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 172/348] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:48.665356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492f7b7-3f04-407e-b0dd-02678c3a7116 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.665825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492f7b7-3f04-407e-b0dd-02678c3a7116 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.666250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492f7b7-3f04-407e-b0dd-02678c3a7116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.694022 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6492f7b7-3f04-407e-b0dd-02678c3a7116 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:48.695547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6492f7b7-3f04-407e-b0dd-02678c3a7116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:48.702016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f5195-6f8b-4c3f-8de8-2bab0ea6039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:48.703032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f5195-6f8b-4c3f-8de8-2bab0ea6039f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.703480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f5195-6f8b-4c3f-8de8-2bab0ea6039f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.704166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f5195-6f8b-4c3f-8de8-2bab0ea6039f None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.713432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f5195-6f8b-4c3f-8de8-2bab0ea6039f None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:48.715507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f5195-6f8b-4c3f-8de8-2bab0ea6039f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:48.717337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 177/349] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:48 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:48.734718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f5195-6f8b-4c3f-8de8-2bab0ea6039f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:48.809621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 173/350] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:48.811974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ffd39-576a-4c71-b0bf-54f8d055173d None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.812494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ffd39-576a-4c71-b0bf-54f8d055173d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.812999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ffd39-576a-4c71-b0bf-54f8d055173d None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.814901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 174/351] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:48.816354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833ffb4-7218-431d-8d2c-69d4fb889200 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.816949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833ffb4-7218-431d-8d2c-69d4fb889200 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.817423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833ffb4-7218-431d-8d2c-69d4fb889200 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.822840 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 175/352] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:48.827700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39f344-5968-46a7-874e-6924b20f994b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.828289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39f344-5968-46a7-874e-6924b20f994b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.828796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39f344-5968-46a7-874e-6924b20f994b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.847285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5d39f344-5968-46a7-874e-6924b20f994b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:48.848913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5d39f344-5968-46a7-874e-6924b20f994b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:48.912804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f5195-6f8b-4c3f-8de8-2bab0ea6039f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:48.916327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:48.944086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 176/353] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:48.948584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f610ee-d65b-439a-97c7-04d33ba82191 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.948584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f610ee-d65b-439a-97c7-04d33ba82191 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.948584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f610ee-d65b-439a-97c7-04d33ba82191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.960635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:48.960883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9a505-f0dd-4eef-96c9-ea3b5f949bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:48.961749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9a505-f0dd-4eef-96c9-ea3b5f949bfe None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:48.962033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9a505-f0dd-4eef-96c9-ea3b5f949bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:48.962306 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9a505-f0dd-4eef-96c9-ea3b5f949bfe None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:48.963587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9a505-f0dd-4eef-96c9-ea3b5f949bfe None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:48.964384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d4f610ee-d65b-439a-97c7-04d33ba82191 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:48.965781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d4f610ee-d65b-439a-97c7-04d33ba82191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:48.966122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9a505-f0dd-4eef-96c9-ea3b5f949bfe None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:48.968248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 178/354] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:48 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:48.969932 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9a505-f0dd-4eef-96c9-ea3b5f949bfe None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:49.062821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 177/355] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:49.064126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f610ee-d65b-439a-97c7-04d33ba82191 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:49.120805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9a505-f0dd-4eef-96c9-ea3b5f949bfe None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:49.163694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b425c-0f51-4b82-a2ab-eaefb9490799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:49.164557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b425c-0f51-4b82-a2ab-eaefb9490799 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.164845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b425c-0f51-4b82-a2ab-eaefb9490799 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.165117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b425c-0f51-4b82-a2ab-eaefb9490799 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:49.179785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b425c-0f51-4b82-a2ab-eaefb9490799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-264b425c-0f51-4b82-a2ab-eaefb9490799 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:49.189022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:49.189941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:49.191190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 179/356] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:49.191854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b425c-0f51-4b82-a2ab-eaefb9490799 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:49.195082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f610ee-d65b-439a-97c7-04d33ba82191 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:49.232729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:49.244625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7d6a1-5aae-44fd-a1aa-f94d1cb2e776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:49.246315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7d6a1-5aae-44fd-a1aa-f94d1cb2e776 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.246802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7d6a1-5aae-44fd-a1aa-f94d1cb2e776 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.247951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7d6a1-5aae-44fd-a1aa-f94d1cb2e776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:49.249320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7d6a1-5aae-44fd-a1aa-f94d1cb2e776 None admin] RBAC: Adding query filter params (group.id=3794601340474cd89c8a29d6a289089e, role.id=110c20a9ff1e455d8c43ffdfbdd1447b, scope.project.id=380cd1f0793045d280c092a275d9666a) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:49.249642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7d6a1-5aae-44fd-a1aa-f94d1cb2e776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:49.251109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7d6a1-5aae-44fd-a1aa-f94d1cb2e776 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:49.253344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:49.262186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 178/357] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jul 21 17:20:49 2022] GET /identity/v3/role_assignments?group.id=3794601340474cd89c8a29d6a289089e&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=110c20a9ff1e455d8c43ffdfbdd1447b => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:49.262186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7d6a1-5aae-44fd-a1aa-f94d1cb2e776 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:49.375209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b425c-0f51-4b82-a2ab-eaefb9490799 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:49.403939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7d6a1-5aae-44fd-a1aa-f94d1cb2e776 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:49.427724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413990c-1766-46d4-b51b-5ce0656300e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:49.428578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413990c-1766-46d4-b51b-5ce0656300e0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.428860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413990c-1766-46d4-b51b-5ce0656300e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.429136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413990c-1766-46d4-b51b-5ce0656300e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:49.444787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413990c-1766-46d4-b51b-5ce0656300e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:49.446287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137277c4-e157-4877-a0d1-071728f878d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:49.448588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137277c4-e157-4877-a0d1-071728f878d4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.448588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137277c4-e157-4877-a0d1-071728f878d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.448588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137277c4-e157-4877-a0d1-071728f878d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-5413990c-1766-46d4-b51b-5ce0656300e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:49.455070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:49.455954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:49.457263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 180/358] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:49.458086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413990c-1766-46d4-b51b-5ce0656300e0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:49.467616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137277c4-e157-4877-a0d1-071728f878d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-137277c4-e157-4877-a0d1-071728f878d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:49.475759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:49.476702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 179/359] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:49.478397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0259d-e02a-44d2-afb5-30a17db0a456 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.480742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0259d-e02a-44d2-afb5-30a17db0a456 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.480742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0259d-e02a-44d2-afb5-30a17db0a456 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:49.481071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 180/360] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:49.481761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0259d-e02a-44d2-afb5-30a17db0a456 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:49.559510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:49.559698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:49.579450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413990c-1766-46d4-b51b-5ce0656300e0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:49.610854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0259d-e02a-44d2-afb5-30a17db0a456 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:49.616227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49f41d-0461-4869-8a33-d3db9c0b384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:49.616931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f41d-0461-4869-8a33-d3db9c0b384c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.617211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f41d-0461-4869-8a33-d3db9c0b384c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.617487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f41d-0461-4869-8a33-d3db9c0b384c None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:49.618235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49f41d-0461-4869-8a33-d3db9c0b384c None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:49.618539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49f41d-0461-4869-8a33-d3db9c0b384c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:49.620628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49f41d-0461-4869-8a33-d3db9c0b384c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:49.630470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 181/361] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:49.631203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49f41d-0461-4869-8a33-d3db9c0b384c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:49.649478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229b355-9c4c-432a-9b9b-eac154cfa937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:49.650340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b355-9c4c-432a-9b9b-eac154cfa937 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.650727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b355-9c4c-432a-9b9b-eac154cfa937 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.651134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b355-9c4c-432a-9b9b-eac154cfa937 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:49.652013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229b355-9c4c-432a-9b9b-eac154cfa937 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:49.652606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229b355-9c4c-432a-9b9b-eac154cfa937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:49.654104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229b355-9c4c-432a-9b9b-eac154cfa937 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:49.665098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 181/362] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:49.666445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115eac2c-7705-466c-891d-8711a6912bfb None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.667246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115eac2c-7705-466c-891d-8711a6912bfb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.667750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115eac2c-7705-466c-891d-8711a6912bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:49.691195 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-115eac2c-7705-466c-891d-8711a6912bfb None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:49.692433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-115eac2c-7705-466c-891d-8711a6912bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:49.739873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49f41d-0461-4869-8a33-d3db9c0b384c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:49.767772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 182/363] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:49.768647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115eac2c-7705-466c-891d-8711a6912bfb None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:49.777367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db38a1-e26e-41aa-b6a0-b670b26556a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:49.778154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db38a1-e26e-41aa-b6a0-b670b26556a7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.778440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db38a1-e26e-41aa-b6a0-b670b26556a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.778710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db38a1-e26e-41aa-b6a0-b670b26556a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:49.779459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db38a1-e26e-41aa-b6a0-b670b26556a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:49.779761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db38a1-e26e-41aa-b6a0-b670b26556a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:49.781167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db38a1-e26e-41aa-b6a0-b670b26556a7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:49.790088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 182/364] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:49.790707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db38a1-e26e-41aa-b6a0-b670b26556a7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:49.810831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:49.847464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:49.903367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db38a1-e26e-41aa-b6a0-b670b26556a7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:49.903554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115eac2c-7705-466c-891d-8711a6912bfb None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:49.940927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cc938-2150-43a6-8c81-74a1136a898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:49.940927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cc938-2150-43a6-8c81-74a1136a898d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.941377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cc938-2150-43a6-8c81-74a1136a898d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.941377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cc938-2150-43a6-8c81-74a1136a898d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:49.952005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409f554-660c-4be2-82c7-32deb99d1494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:49.952694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409f554-660c-4be2-82c7-32deb99d1494 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:49.952978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409f554-660c-4be2-82c7-32deb99d1494 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:49.953254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409f554-660c-4be2-82c7-32deb99d1494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:50.014061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cc938-2150-43a6-8c81-74a1136a898d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:50.015524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cc938-2150-43a6-8c81-74a1136a898d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:50.028436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409f554-660c-4be2-82c7-32deb99d1494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:50.029905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409f554-660c-4be2-82c7-32deb99d1494 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:50.033043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:50.081731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:50.123048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 183/365] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:50.123922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cc938-2150-43a6-8c81-74a1136a898d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:50.142455 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 183/366] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:50.143387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409f554-660c-4be2-82c7-32deb99d1494 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:50.227832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cc938-2150-43a6-8c81-74a1136a898d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:50.254807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409f554-660c-4be2-82c7-32deb99d1494 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:50.256804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:50.270925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9561d79-e1bf-4299-98ac-a0733b834869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:50.271820 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9561d79-e1bf-4299-98ac-a0733b834869 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:50.272098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9561d79-e1bf-4299-98ac-a0733b834869 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:50.272403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9561d79-e1bf-4299-98ac-a0733b834869 None admin] PATH_INFO: `/v3/users/5f6f35c851064dfd8151818af98136f9` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:50.273646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9561d79-e1bf-4299-98ac-a0733b834869 None admin] RBAC: Authorizing `identity:get_user(user_id=5f6f35c851064dfd8151818af98136f9)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:50.275115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9561d79-e1bf-4299-98ac-a0733b834869 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:50.278166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 184/367] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:20:50 2022] GET /identity/v3/users/5f6f35c851064dfd8151818af98136f9 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:50.278980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9561d79-e1bf-4299-98ac-a0733b834869 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:50.298691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57a293-96ff-4a6e-bca7-a0258b8df68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:50.299497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57a293-96ff-4a6e-bca7-a0258b8df68e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:50.299778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57a293-96ff-4a6e-bca7-a0258b8df68e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:50.300051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57a293-96ff-4a6e-bca7-a0258b8df68e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:50.312161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57a293-96ff-4a6e-bca7-a0258b8df68e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-bc57a293-96ff-4a6e-bca7-a0258b8df68e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:50.320977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:50.321862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:50.322683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 184/368] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:50.323369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57a293-96ff-4a6e-bca7-a0258b8df68e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:50.336444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:50.412691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9561d79-e1bf-4299-98ac-a0733b834869 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:50.435153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57a293-96ff-4a6e-bca7-a0258b8df68e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:50.465080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27906d-ca95-45d4-9b1d-ca6580340f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:50.466922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27906d-ca95-45d4-9b1d-ca6580340f51 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:50.467435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27906d-ca95-45d4-9b1d-ca6580340f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:50.468151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27906d-ca95-45d4-9b1d-ca6580340f51 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:50.470259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27906d-ca95-45d4-9b1d-ca6580340f51 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:50.472215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27906d-ca95-45d4-9b1d-ca6580340f51 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:50.474473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 185/369] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:50 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:50.474651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcbd2d-ea3b-4015-9cb3-4adc01f01aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:50.475474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcbd2d-ea3b-4015-9cb3-4adc01f01aaa None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:50.475760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcbd2d-ea3b-4015-9cb3-4adc01f01aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:50.476040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcbd2d-ea3b-4015-9cb3-4adc01f01aaa None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:50.476261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27906d-ca95-45d4-9b1d-ca6580340f51 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:50.480375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcbd2d-ea3b-4015-9cb3-4adc01f01aaa None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:50.480375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcbd2d-ea3b-4015-9cb3-4adc01f01aaa None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:50.481169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 185/370] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:50 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:50.482587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcbd2d-ea3b-4015-9cb3-4adc01f01aaa None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:50.555868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:50.582739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:50.607329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27906d-ca95-45d4-9b1d-ca6580340f51 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:50.611521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcbd2d-ea3b-4015-9cb3-4adc01f01aaa None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:50.645373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3567e16-61ed-4362-8bb8-02bfb99c8119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:50.646118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3567e16-61ed-4362-8bb8-02bfb99c8119 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:50.646406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3567e16-61ed-4362-8bb8-02bfb99c8119 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:50.646764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3567e16-61ed-4362-8bb8-02bfb99c8119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:50.647795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3567e16-61ed-4362-8bb8-02bfb99c8119 None admin] RBAC: Adding query filter params (role.id=feffc80bdc3641a59c6d5d545c7fce94, user.id=5f6f35c851064dfd8151818af98136f9, scope.system=all) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:50.648119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3567e16-61ed-4362-8bb8-02bfb99c8119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:50.649670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21593d20-cebb-4a0a-84f7-e6b7dce05077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:50.649808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3567e16-61ed-4362-8bb8-02bfb99c8119 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:50.650688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21593d20-cebb-4a0a-84f7-e6b7dce05077 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:50.651697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21593d20-cebb-4a0a-84f7-e6b7dce05077 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:50.652255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21593d20-cebb-4a0a-84f7-e6b7dce05077 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:50.653378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21593d20-cebb-4a0a-84f7-e6b7dce05077 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:50.653893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21593d20-cebb-4a0a-84f7-e6b7dce05077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:50.655674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21593d20-cebb-4a0a-84f7-e6b7dce05077 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:50.664415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 186/371] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Jul 21 17:20:50 2022] GET /identity/v3/role_assignments?user.id=5f6f35c851064dfd8151818af98136f9&scope.system=all&role.id=feffc80bdc3641a59c6d5d545c7fce94 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:50.665462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3567e16-61ed-4362-8bb8-02bfb99c8119 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:50.665914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 186/372] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:50.667512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21593d20-cebb-4a0a-84f7-e6b7dce05077 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:50.785432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3567e16-61ed-4362-8bb8-02bfb99c8119 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:50.790900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21593d20-cebb-4a0a-84f7-e6b7dce05077 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:50.809436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:50.835034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097bb0f-fd23-4708-81e6-0f7486d8a835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:50.837891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097bb0f-fd23-4708-81e6-0f7486d8a835 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:50.838459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097bb0f-fd23-4708-81e6-0f7486d8a835 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:50.839444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097bb0f-fd23-4708-81e6-0f7486d8a835 None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:50.846043 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097bb0f-fd23-4708-81e6-0f7486d8a835 None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:50.847961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097bb0f-fd23-4708-81e6-0f7486d8a835 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:50.852709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 187/373] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:50 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:50.855210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097bb0f-fd23-4708-81e6-0f7486d8a835 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:50.867001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f3f6-6278-4af9-9cf9-21273532dc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:50.868274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f3f6-6278-4af9-9cf9-21273532dc5c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:50.868860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f3f6-6278-4af9-9cf9-21273532dc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:50.869378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f3f6-6278-4af9-9cf9-21273532dc5c None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:50.871295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827f3f6-6278-4af9-9cf9-21273532dc5c None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:50.874988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827f3f6-6278-4af9-9cf9-21273532dc5c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:50.877650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 187/374] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:50 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:50.882175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f3f6-6278-4af9-9cf9-21273532dc5c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:50.884286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:51.014376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097bb0f-fd23-4708-81e6-0f7486d8a835 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:51.039983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f3f6-6278-4af9-9cf9-21273532dc5c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:51.051469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402bf81-ad2a-4307-8b17-d33ee5507213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:51.052136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402bf81-ad2a-4307-8b17-d33ee5507213 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:51.052458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402bf81-ad2a-4307-8b17-d33ee5507213 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:51.052739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402bf81-ad2a-4307-8b17-d33ee5507213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:51.079630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1eb05-471a-4baa-80cb-8de1c2725552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:51.080412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1eb05-471a-4baa-80cb-8de1c2725552 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:51.080699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1eb05-471a-4baa-80cb-8de1c2725552 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:51.081059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1eb05-471a-4baa-80cb-8de1c2725552 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:51.081893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1eb05-471a-4baa-80cb-8de1c2725552 None admin] RBAC: Adding query filter params (group.id=3794601340474cd89c8a29d6a289089e, role.id=110c20a9ff1e455d8c43ffdfbdd1447b, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:51.082201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1eb05-471a-4baa-80cb-8de1c2725552 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:51.083740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1eb05-471a-4baa-80cb-8de1c2725552 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:51.090893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 188/375] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jul 21 17:20:50 2022] GET /identity/v3/role_assignments?group.id=3794601340474cd89c8a29d6a289089e&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=110c20a9ff1e455d8c43ffdfbdd1447b => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:51.091841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1eb05-471a-4baa-80cb-8de1c2725552 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:51.132184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402bf81-ad2a-4307-8b17-d33ee5507213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:51.133755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402bf81-ad2a-4307-8b17-d33ee5507213 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:51.142164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:51.146558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:51.209014 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1eb05-471a-4baa-80cb-8de1c2725552 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:51.250366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 188/376] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:51.251914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967ce13-e0af-401a-bed2-9f95e1359774 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:51.252099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86167a0a-8766-41e6-877e-29dadc831e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:51.252384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967ce13-e0af-401a-bed2-9f95e1359774 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:51.252741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967ce13-e0af-401a-bed2-9f95e1359774 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:51.252859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86167a0a-8766-41e6-877e-29dadc831e0b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:51.253109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86167a0a-8766-41e6-877e-29dadc831e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:51.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86167a0a-8766-41e6-877e-29dadc831e0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:51.254218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86167a0a-8766-41e6-877e-29dadc831e0b None admin] RBAC: Adding query filter params (group.id=3794601340474cd89c8a29d6a289089e, role.id=d2c2845de1d548bb9cfb46f70619b3f8, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:51.254406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 189/377] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:51.254524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86167a0a-8766-41e6-877e-29dadc831e0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:51.255979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86167a0a-8766-41e6-877e-29dadc831e0b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:51.258254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967ce13-e0af-401a-bed2-9f95e1359774 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:51.264890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 189/378] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jul 21 17:20:51 2022] GET /identity/v3/role_assignments?group.id=3794601340474cd89c8a29d6a289089e&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:51.266390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dc34d-06af-4d51-a26d-5af3e146bb71 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:51.266683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dc34d-06af-4d51-a26d-5af3e146bb71 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:51.266964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dc34d-06af-4d51-a26d-5af3e146bb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:51.290709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0d2dc34d-06af-4d51-a26d-5af3e146bb71 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:51.291991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0d2dc34d-06af-4d51-a26d-5af3e146bb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:51.397534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 190/379] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:51.413095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967ce13-e0af-401a-bed2-9f95e1359774 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:51.424832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2dc34d-06af-4d51-a26d-5af3e146bb71 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:51.427245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:51.486267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1888d-78a9-4ba8-a0f8-9bf4f624596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:51.487150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1888d-78a9-4ba8-a0f8-9bf4f624596d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:51.487434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1888d-78a9-4ba8-a0f8-9bf4f624596d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:51.487713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1888d-78a9-4ba8-a0f8-9bf4f624596d None admin] PATH_INFO: `/v3/projects/ee47bf64e42844579243da3bda418cc7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:51.491942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1888d-78a9-4ba8-a0f8-9bf4f624596d None admin] RBAC: Authorizing `identity:get_project(project_id=ee47bf64e42844579243da3bda418cc7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:51.493435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1888d-78a9-4ba8-a0f8-9bf4f624596d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:51.496153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 190/380] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:51 2022] GET /identity/v3/projects/ee47bf64e42844579243da3bda418cc7 => generated 344 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:51.499877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:51.502750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1888d-78a9-4ba8-a0f8-9bf4f624596d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:51.654279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2dc34d-06af-4d51-a26d-5af3e146bb71 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:51.664648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1888d-78a9-4ba8-a0f8-9bf4f624596d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:51.705040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81540407-5f7c-437f-959f-bbb526eff91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:51.705912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81540407-5f7c-437f-959f-bbb526eff91a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:51.706195 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81540407-5f7c-437f-959f-bbb526eff91a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:51.706469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81540407-5f7c-437f-959f-bbb526eff91a None admin] PATH_INFO: `/v3/groups/a91f682189014c91a1fd5df1ab74f15d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:51.716764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81540407-5f7c-437f-959f-bbb526eff91a None admin] RBAC: Authorizing `identity:get_group(group_id=a91f682189014c91a1fd5df1ab74f15d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:51.718235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81540407-5f7c-437f-959f-bbb526eff91a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:51.719974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 191/381] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:51 2022] GET /identity/v3/groups/a91f682189014c91a1fd5df1ab74f15d => generated 229 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:51.727385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81540407-5f7c-437f-959f-bbb526eff91a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:51.742396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac41412-b21b-4664-883c-a4e730be3341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:51.743206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41412-b21b-4664-883c-a4e730be3341 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:51.743484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41412-b21b-4664-883c-a4e730be3341 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:51.743757 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41412-b21b-4664-883c-a4e730be3341 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:51.760927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac41412-b21b-4664-883c-a4e730be3341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2ac41412-b21b-4664-883c-a4e730be3341 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:51.769892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:51.770795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:51.771585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:51.774629 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 191/382] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:51.784086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac41412-b21b-4664-883c-a4e730be3341 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:51.819519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:51.928626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81540407-5f7c-437f-959f-bbb526eff91a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:51.948709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:51.957588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac41412-b21b-4664-883c-a4e730be3341 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:51.997920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678da4d2-32b3-47c9-ab0f-761bd73e6016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:52.000283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678da4d2-32b3-47c9-ab0f-761bd73e6016 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:52.000283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678da4d2-32b3-47c9-ab0f-761bd73e6016 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:52.000283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678da4d2-32b3-47c9-ab0f-761bd73e6016 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:52.000283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678da4d2-32b3-47c9-ab0f-761bd73e6016 None admin] RBAC: Adding query filter params (group.id=a91f682189014c91a1fd5df1ab74f15d, role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=ee47bf64e42844579243da3bda418cc7) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:52.000600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678da4d2-32b3-47c9-ab0f-761bd73e6016 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:52.001963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678da4d2-32b3-47c9-ab0f-761bd73e6016 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:52.002605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e39d8e-8051-4e13-bb0c-3242322dd326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:52.003527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e39d8e-8051-4e13-bb0c-3242322dd326 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:52.004075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e39d8e-8051-4e13-bb0c-3242322dd326 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:52.004582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e39d8e-8051-4e13-bb0c-3242322dd326 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:52.005549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e39d8e-8051-4e13-bb0c-3242322dd326 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:52.006073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e39d8e-8051-4e13-bb0c-3242322dd326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:52.007642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e39d8e-8051-4e13-bb0c-3242322dd326 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:52.014582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 192/383] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jul 21 17:20:51 2022] GET /identity/v3/role_assignments?group.id=a91f682189014c91a1fd5df1ab74f15d&scope.project.id=ee47bf64e42844579243da3bda418cc7&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 260 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:52.019664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 192/384] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:52.025844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678da4d2-32b3-47c9-ab0f-761bd73e6016 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:52.106767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c9755-e631-4d88-b287-04e52ffe3da6 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:52.107862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c9755-e631-4d88-b287-04e52ffe3da6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:52.108384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c9755-e631-4d88-b287-04e52ffe3da6 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:52.110112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 193/385] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:52.116081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a52e73-a3d8-41f5-8cc6-60ced90b894d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:52.116723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a52e73-a3d8-41f5-8cc6-60ced90b894d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:52.117223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a52e73-a3d8-41f5-8cc6-60ced90b894d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:52.144539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d0a52e73-a3d8-41f5-8cc6-60ced90b894d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:52.147354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d0a52e73-a3d8-41f5-8cc6-60ced90b894d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:52.192161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678da4d2-32b3-47c9-ab0f-761bd73e6016 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:52.196374 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:52.251708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba4034-203a-40d8-86ef-d38f58fd5914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:52.252627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba4034-203a-40d8-86ef-d38f58fd5914 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:52.252904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba4034-203a-40d8-86ef-d38f58fd5914 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:52.253591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba4034-203a-40d8-86ef-d38f58fd5914 None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:52.254798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba4034-203a-40d8-86ef-d38f58fd5914 None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:52.256911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba4034-203a-40d8-86ef-d38f58fd5914 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:52.258872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 193/386] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:52 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:52.264879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba4034-203a-40d8-86ef-d38f58fd5914 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:52.286896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 194/387] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:52.312212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a52e73-a3d8-41f5-8cc6-60ced90b894d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:52.409368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:52.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a52e73-a3d8-41f5-8cc6-60ced90b894d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:52.427935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba4034-203a-40d8-86ef-d38f58fd5914 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:52.460057 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9544fa-3d6a-48f3-ae8a-3a9f3a8fae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:52.460830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9544fa-3d6a-48f3-ae8a-3a9f3a8fae2a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:52.461230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9544fa-3d6a-48f3-ae8a-3a9f3a8fae2a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:52.461511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9544fa-3d6a-48f3-ae8a-3a9f3a8fae2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:52.473245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9544fa-3d6a-48f3-ae8a-3a9f3a8fae2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-cd9544fa-3d6a-48f3-ae8a-3a9f3a8fae2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:52.481692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:52.482661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:52.483504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:52.487745 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 195/388] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:52.488961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9544fa-3d6a-48f3-ae8a-3a9f3a8fae2a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:52.506014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d501ccf-fe26-41bd-8bf1-5244e4caf27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:52.506949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d501ccf-fe26-41bd-8bf1-5244e4caf27f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:52.507228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d501ccf-fe26-41bd-8bf1-5244e4caf27f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:52.507503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d501ccf-fe26-41bd-8bf1-5244e4caf27f None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:52.509003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d501ccf-fe26-41bd-8bf1-5244e4caf27f None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:52.511097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d501ccf-fe26-41bd-8bf1-5244e4caf27f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:52.513323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 194/389] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:52 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:52.519055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d501ccf-fe26-41bd-8bf1-5244e4caf27f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:52.597064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9544fa-3d6a-48f3-ae8a-3a9f3a8fae2a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:52.635735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d36e4-d28b-4aeb-8128-3d1eb2e6e56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:52.636414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:52.637622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d36e4-d28b-4aeb-8128-3d1eb2e6e56a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:52.637907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d36e4-d28b-4aeb-8128-3d1eb2e6e56a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:52.638363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d36e4-d28b-4aeb-8128-3d1eb2e6e56a None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:52.638942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d36e4-d28b-4aeb-8128-3d1eb2e6e56a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:52.639258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d36e4-d28b-4aeb-8128-3d1eb2e6e56a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:52.641597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d36e4-d28b-4aeb-8128-3d1eb2e6e56a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:52.651020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 196/390] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:52.656780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d36e4-d28b-4aeb-8128-3d1eb2e6e56a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:52.692710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:52.779661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d501ccf-fe26-41bd-8bf1-5244e4caf27f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:52.826938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d36e4-d28b-4aeb-8128-3d1eb2e6e56a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:52.840941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2df95-e9b0-4dac-b8de-a21b7f31a05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:52.841848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2df95-e9b0-4dac-b8de-a21b7f31a05f None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:52.842132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2df95-e9b0-4dac-b8de-a21b7f31a05f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:52.842408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2df95-e9b0-4dac-b8de-a21b7f31a05f None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a/groups/3794601340474cd89c8a29d6a289089e/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:52.846222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2df95-e9b0-4dac-b8de-a21b7f31a05f None admin] RBAC: Authorizing `identity:create_grant(project_id=380cd1f0793045d280c092a275d9666a, group_id=3794601340474cd89c8a29d6a289089e, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:52.847686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2df95-e9b0-4dac-b8de-a21b7f31a05f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:52.871878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 195/391] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Jul 21 17:20:52 2022] PUT /identity/v3/projects/380cd1f0793045d280c092a275d9666a/groups/3794601340474cd89c8a29d6a289089e/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:52.899592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6b278-1b2e-4930-8f16-b33784f15e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:52.900564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6b278-1b2e-4930-8f16-b33784f15e82 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:52.900848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6b278-1b2e-4930-8f16-b33784f15e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:52.901661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6b278-1b2e-4930-8f16-b33784f15e82 None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:52.904281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6b278-1b2e-4930-8f16-b33784f15e82 None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:52.905780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6b278-1b2e-4930-8f16-b33784f15e82 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:52.907719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 197/392] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:52 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:52.919586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2df95-e9b0-4dac-b8de-a21b7f31a05f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:53.044200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:53.048006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2df95-e9b0-4dac-b8de-a21b7f31a05f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:53.096037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f46b1-c3e0-451e-807e-0bbb055a661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:53.096950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f46b1-c3e0-451e-807e-0bbb055a661e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:53.097234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f46b1-c3e0-451e-807e-0bbb055a661e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:53.097507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f46b1-c3e0-451e-807e-0bbb055a661e None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:53.098815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f46b1-c3e0-451e-807e-0bbb055a661e None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:53.100354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f46b1-c3e0-451e-807e-0bbb055a661e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:53.102347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 196/393] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:52 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:53.108981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6b278-1b2e-4930-8f16-b33784f15e82 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:53.200132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:53.223868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6b278-1b2e-4930-8f16-b33784f15e82 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:53.262011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84e2c9-fbe0-4472-ad19-244fa6c2383d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:53.262903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e2c9-fbe0-4472-ad19-244fa6c2383d None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:53.263186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e2c9-fbe0-4472-ad19-244fa6c2383d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:53.263463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e2c9-fbe0-4472-ad19-244fa6c2383d None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a/groups/3794601340474cd89c8a29d6a289089e/roles/110c20a9ff1e455d8c43ffdfbdd1447b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:53.265739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84e2c9-fbe0-4472-ad19-244fa6c2383d None admin] RBAC: Authorizing `identity:create_grant(project_id=380cd1f0793045d280c092a275d9666a, group_id=3794601340474cd89c8a29d6a289089e, role_id=110c20a9ff1e455d8c43ffdfbdd1447b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:53.267190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84e2c9-fbe0-4472-ad19-244fa6c2383d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:53.285347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 198/394] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Jul 21 17:20:53 2022] PUT /identity/v3/projects/380cd1f0793045d280c092a275d9666a/groups/3794601340474cd89c8a29d6a289089e/roles/110c20a9ff1e455d8c43ffdfbdd1447b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:53.690452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cb30e-f19c-40a6-bf0f-0ab3d07112c1 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:53.690826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cb30e-f19c-40a6-bf0f-0ab3d07112c1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:53.691073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cb30e-f19c-40a6-bf0f-0ab3d07112c1 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:53.692627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 197/395] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:53.697621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eae49-0cd3-489b-b2a7-bda4f94911ed None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:53.698488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eae49-0cd3-489b-b2a7-bda4f94911ed None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:53.698776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eae49-0cd3-489b-b2a7-bda4f94911ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:53.722737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-449eae49-0cd3-489b-b2a7-bda4f94911ed None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:53.723922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-449eae49-0cd3-489b-b2a7-bda4f94911ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:53.810836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 199/396] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:53.834518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cb30e-f19c-40a6-bf0f-0ab3d07112c1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:53.966080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cb30e-f19c-40a6-bf0f-0ab3d07112c1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:54.005577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086f648-46e0-4557-9434-a204d579fd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:54.006396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086f648-46e0-4557-9434-a204d579fd28 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.006674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086f648-46e0-4557-9434-a204d579fd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.006944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086f648-46e0-4557-9434-a204d579fd28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.020175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086f648-46e0-4557-9434-a204d579fd28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8086f648-46e0-4557-9434-a204d579fd28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:54.029109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:54.029995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:54.030791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 198/397] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:54.036418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449eae49-0cd3-489b-b2a7-bda4f94911ed None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:54.107276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:54.142144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449eae49-0cd3-489b-b2a7-bda4f94911ed None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:54.177979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7aa76-4bf5-41e6-9c07-ef06c465e19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:54.178656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7aa76-4bf5-41e6-9c07-ef06c465e19d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.178986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7aa76-4bf5-41e6-9c07-ef06c465e19d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.179212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7aa76-4bf5-41e6-9c07-ef06c465e19d None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.179963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7aa76-4bf5-41e6-9c07-ef06c465e19d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:54.180275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7aa76-4bf5-41e6-9c07-ef06c465e19d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:54.181678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7aa76-4bf5-41e6-9c07-ef06c465e19d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:54.190801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 200/398] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:54.195996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086f648-46e0-4557-9434-a204d579fd28 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:54.227098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:54.248878 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75a6f8-32e6-4c64-8574-92fed709d863 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.249324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75a6f8-32e6-4c64-8574-92fed709d863 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.249722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75a6f8-32e6-4c64-8574-92fed709d863 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.251868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 201/399] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:54.258131 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213a9f6-08cc-4870-acb3-f3d4ec5e2962 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.258987 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213a9f6-08cc-4870-acb3-f3d4ec5e2962 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.259368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213a9f6-08cc-4870-acb3-f3d4ec5e2962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.281260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0213a9f6-08cc-4870-acb3-f3d4ec5e2962 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:54.281772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0213a9f6-08cc-4870-acb3-f3d4ec5e2962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:54.331787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086f648-46e0-4557-9434-a204d579fd28 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:54.360478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 202/400] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:54.377026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3cd19-1857-44a9-9daa-c5cebbf62b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:54.378188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3cd19-1857-44a9-9daa-c5cebbf62b13 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.378622 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3cd19-1857-44a9-9daa-c5cebbf62b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.379046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3cd19-1857-44a9-9daa-c5cebbf62b13 None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.381104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3cd19-1857-44a9-9daa-c5cebbf62b13 None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:54.382822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3cd19-1857-44a9-9daa-c5cebbf62b13 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:54.385116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 199/401] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:54 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:54.388248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213a9f6-08cc-4870-acb3-f3d4ec5e2962 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:54.395375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3cd19-1857-44a9-9daa-c5cebbf62b13 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:54.507583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:54.576931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213a9f6-08cc-4870-acb3-f3d4ec5e2962 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:54.592695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:54.597725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3cd19-1857-44a9-9daa-c5cebbf62b13 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:54.646304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67294e43-7680-47d3-ac53-96162e9ef489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:54.647192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67294e43-7680-47d3-ac53-96162e9ef489 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.647444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67294e43-7680-47d3-ac53-96162e9ef489 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.647721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67294e43-7680-47d3-ac53-96162e9ef489 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.662929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67294e43-7680-47d3-ac53-96162e9ef489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:54.666294 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609506e-ff5d-4c79-a42b-4977ce5efb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:54.667987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609506e-ff5d-4c79-a42b-4977ce5efb89 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.668650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609506e-ff5d-4c79-a42b-4977ce5efb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.669148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609506e-ff5d-4c79-a42b-4977ce5efb89 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-67294e43-7680-47d3-ac53-96162e9ef489 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:54.672191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:54.673095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:54.673887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:54.675704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609506e-ff5d-4c79-a42b-4977ce5efb89 None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:54.677410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609506e-ff5d-4c79-a42b-4977ce5efb89 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:54.678385 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 203/402] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:54.679605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67294e43-7680-47d3-ac53-96162e9ef489 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:54.679871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 200/403] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:54 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:54.686262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609506e-ff5d-4c79-a42b-4977ce5efb89 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:54.798353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67294e43-7680-47d3-ac53-96162e9ef489 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:54.820076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609506e-ff5d-4c79-a42b-4977ce5efb89 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:54.820890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:54.837770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdc6d3-021c-417d-a5fc-72bf3d21e216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:54.838813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdc6d3-021c-417d-a5fc-72bf3d21e216 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.838813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdc6d3-021c-417d-a5fc-72bf3d21e216 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.839116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdc6d3-021c-417d-a5fc-72bf3d21e216 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.840325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdc6d3-021c-417d-a5fc-72bf3d21e216 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:54.840619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdc6d3-021c-417d-a5fc-72bf3d21e216 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:54.842020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdc6d3-021c-417d-a5fc-72bf3d21e216 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:54.851060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 204/404] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:54.852463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c95b-64fa-43ad-a7c0-7a0c81762ec3 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.852753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c95b-64fa-43ad-a7c0-7a0c81762ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.853029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c95b-64fa-43ad-a7c0-7a0c81762ec3 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.855141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 205/405] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:54.860541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c95b-64fa-43ad-a7c0-7a0c81762ec3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:54.863967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa39c2-bd5f-49a9-90a4-92655fe9f826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:54.865227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa39c2-bd5f-49a9-90a4-92655fe9f826 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.865716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa39c2-bd5f-49a9-90a4-92655fe9f826 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.866193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa39c2-bd5f-49a9-90a4-92655fe9f826 None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/groups/3794601340474cd89c8a29d6a289089e/roles/110c20a9ff1e455d8c43ffdfbdd1447b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.868700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa39c2-bd5f-49a9-90a4-92655fe9f826 None admin] RBAC: Authorizing `identity:create_grant(project_id=4819e5a17eb24efcb8ccbb51a48b30ca, group_id=3794601340474cd89c8a29d6a289089e, role_id=110c20a9ff1e455d8c43ffdfbdd1447b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:54.871865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa39c2-bd5f-49a9-90a4-92655fe9f826 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:54.892396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 201/406] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Jul 21 17:20:54 2022] PUT /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/groups/3794601340474cd89c8a29d6a289089e/roles/110c20a9ff1e455d8c43ffdfbdd1447b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:20:54.894258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a00ce-a973-4ff3-86cf-495d4bfe2e9e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:54.894771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a00ce-a973-4ff3-86cf-495d4bfe2e9e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:54.895355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a00ce-a973-4ff3-86cf-495d4bfe2e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:54.921874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-684a00ce-a973-4ff3-86cf-495d4bfe2e9e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:54.923461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-684a00ce-a973-4ff3-86cf-495d4bfe2e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:54.956742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:55.016523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 202/407] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:55.042383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a00ce-a973-4ff3-86cf-495d4bfe2e9e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:55.103629 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c95b-64fa-43ad-a7c0-7a0c81762ec3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:55.147786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5e242-a1a6-433e-92df-4e6567297111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:55.149478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5e242-a1a6-433e-92df-4e6567297111 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:55.150022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5e242-a1a6-433e-92df-4e6567297111 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:55.150546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5e242-a1a6-433e-92df-4e6567297111 None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:55.152655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5e242-a1a6-433e-92df-4e6567297111 None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:55.154487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5e242-a1a6-433e-92df-4e6567297111 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:55.156929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 206/408] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:54 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:55.163524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5e242-a1a6-433e-92df-4e6567297111 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:55.178445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a00ce-a973-4ff3-86cf-495d4bfe2e9e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:55.216446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae00fb1-f5d1-435d-af40-735dcf863309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:55.217233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae00fb1-f5d1-435d-af40-735dcf863309 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:55.217506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae00fb1-f5d1-435d-af40-735dcf863309 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:55.217783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae00fb1-f5d1-435d-af40-735dcf863309 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:55.226961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:55.228979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae00fb1-f5d1-435d-af40-735dcf863309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-0ae00fb1-f5d1-435d-af40-735dcf863309 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:55.238525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:55.239444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:55.240231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 203/409] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:55.246217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae00fb1-f5d1-435d-af40-735dcf863309 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:55.263786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:55.289032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5e242-a1a6-433e-92df-4e6567297111 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:55.328359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204ecd5-1688-4398-adcc-b083d2ccfa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:55.329706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204ecd5-1688-4398-adcc-b083d2ccfa7e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:55.330195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204ecd5-1688-4398-adcc-b083d2ccfa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:55.330682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204ecd5-1688-4398-adcc-b083d2ccfa7e None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:55.332207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204ecd5-1688-4398-adcc-b083d2ccfa7e None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:55.334130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204ecd5-1688-4398-adcc-b083d2ccfa7e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:55.336480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 207/410] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:55 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:55.342872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204ecd5-1688-4398-adcc-b083d2ccfa7e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:55.355557 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae00fb1-f5d1-435d-af40-735dcf863309 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:55.391663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40929c06-e8e4-40a4-bc1d-8b0111eb0425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:55.392413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40929c06-e8e4-40a4-bc1d-8b0111eb0425 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:55.392659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40929c06-e8e4-40a4-bc1d-8b0111eb0425 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:55.392921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40929c06-e8e4-40a4-bc1d-8b0111eb0425 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:55.393692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40929c06-e8e4-40a4-bc1d-8b0111eb0425 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:55.394004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40929c06-e8e4-40a4-bc1d-8b0111eb0425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:55.395404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40929c06-e8e4-40a4-bc1d-8b0111eb0425 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:55.405618 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 204/411] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:55.411225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40929c06-e8e4-40a4-bc1d-8b0111eb0425 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:55.476818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204ecd5-1688-4398-adcc-b083d2ccfa7e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:55.493010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:55.497160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:55.518394 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d53f1-a1aa-4999-86c4-f739a42f54bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:55.519648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d53f1-a1aa-4999-86c4-f739a42f54bf None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:55.520132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d53f1-a1aa-4999-86c4-f739a42f54bf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:55.520656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d53f1-a1aa-4999-86c4-f739a42f54bf None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/groups/3794601340474cd89c8a29d6a289089e/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:55.520760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40929c06-e8e4-40a4-bc1d-8b0111eb0425 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:55.523229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d53f1-a1aa-4999-86c4-f739a42f54bf None admin] RBAC: Authorizing `identity:create_grant(project_id=4819e5a17eb24efcb8ccbb51a48b30ca, group_id=3794601340474cd89c8a29d6a289089e, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:55.524934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d53f1-a1aa-4999-86c4-f739a42f54bf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:55.542234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 208/412] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Jul 21 17:20:55 2022] PUT /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca/groups/3794601340474cd89c8a29d6a289089e/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:55.562002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e80586-6da5-479c-b4e4-b4202916a1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:55.562881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e80586-6da5-479c-b4e4-b4202916a1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:55.563169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e80586-6da5-479c-b4e4-b4202916a1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:55.563448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e80586-6da5-479c-b4e4-b4202916a1b2 None admin] PATH_INFO: `/v3/groups/a91f682189014c91a1fd5df1ab74f15d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:55.564711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e80586-6da5-479c-b4e4-b4202916a1b2 None admin] RBAC: Authorizing `identity:get_group(group_id=a91f682189014c91a1fd5df1ab74f15d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:55.566183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e80586-6da5-479c-b4e4-b4202916a1b2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:55.573638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 205/413] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:55 2022] GET /identity/v3/groups/a91f682189014c91a1fd5df1ab74f15d => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:55.575297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d53f1-a1aa-4999-86c4-f739a42f54bf None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:55.706849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d53f1-a1aa-4999-86c4-f739a42f54bf None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:55.747599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bd9ab-0cf0-4749-b179-1076a25241b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:55.749018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bd9ab-0cf0-4749-b179-1076a25241b5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:55.749537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bd9ab-0cf0-4749-b179-1076a25241b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:55.750019 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bd9ab-0cf0-4749-b179-1076a25241b5 None admin] PATH_INFO: `/v3/projects/ee47bf64e42844579243da3bda418cc7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:55.751545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bd9ab-0cf0-4749-b179-1076a25241b5 None admin] RBAC: Authorizing `identity:get_project(project_id=ee47bf64e42844579243da3bda418cc7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:55.753381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bd9ab-0cf0-4749-b179-1076a25241b5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:55.755749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 209/414] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:55 2022] GET /identity/v3/projects/ee47bf64e42844579243da3bda418cc7 => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:55.764348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e80586-6da5-479c-b4e4-b4202916a1b2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:55.883049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e80586-6da5-479c-b4e4-b4202916a1b2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:55.924687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff91a66-4558-4d6d-a382-242054bc83d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:55.925936 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff91a66-4558-4d6d-a382-242054bc83d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:55.926423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff91a66-4558-4d6d-a382-242054bc83d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:55.926922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff91a66-4558-4d6d-a382-242054bc83d3 None admin] PATH_INFO: `/v3/projects/ee47bf64e42844579243da3bda418cc7/groups/a91f682189014c91a1fd5df1ab74f15d/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:55.935210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff91a66-4558-4d6d-a382-242054bc83d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee47bf64e42844579243da3bda418cc7, group_id=a91f682189014c91a1fd5df1ab74f15d, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:55.936940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff91a66-4558-4d6d-a382-242054bc83d3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:55.967797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 206/415] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Jul 21 17:20:55 2022] PUT /identity/v3/projects/ee47bf64e42844579243da3bda418cc7/groups/a91f682189014c91a1fd5df1ab74f15d/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:20:55.990748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f353a0b-be2f-487b-ad9b-eb56db57b665 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:55.991059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f353a0b-be2f-487b-ad9b-eb56db57b665 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:55.991310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f353a0b-be2f-487b-ad9b-eb56db57b665 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:55.993153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 210/416] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:55.999517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb7ee1-d1f9-4d88-a8f9-71ab17ccda16 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:55.999802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb7ee1-d1f9-4d88-a8f9-71ab17ccda16 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:56.000083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb7ee1-d1f9-4d88-a8f9-71ab17ccda16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:56.007242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:56.019853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-41eb7ee1-d1f9-4d88-a8f9-71ab17ccda16 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:56.021797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-41eb7ee1-d1f9-4d88-a8f9-71ab17ccda16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:56.121048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 207/417] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:56.159425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f353a0b-be2f-487b-ad9b-eb56db57b665 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:56.168092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab483a9d-8ec3-4d13-8b94-b60f38ce2d28 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:56.168409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab483a9d-8ec3-4d13-8b94-b60f38ce2d28 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:56.168691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab483a9d-8ec3-4d13-8b94-b60f38ce2d28 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:56.170061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 208/418] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:56.176010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e592ad-8210-432a-b609-a32bb7526ce1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:56.176329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e592ad-8210-432a-b609-a32bb7526ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:56.176602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e592ad-8210-432a-b609-a32bb7526ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:56.198554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b3e592ad-8210-432a-b609-a32bb7526ce1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:56.200426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b3e592ad-8210-432a-b609-a32bb7526ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:56.227258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:56.294004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f353a0b-be2f-487b-ad9b-eb56db57b665 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:56.329282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d86995-8ecc-4a26-991f-34dc06f934e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:56.329903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 209/419] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:56.330326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d86995-8ecc-4a26-991f-34dc06f934e3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:56.330399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d86995-8ecc-4a26-991f-34dc06f934e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:56.332332 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d86995-8ecc-4a26-991f-34dc06f934e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:56.342010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d86995-8ecc-4a26-991f-34dc06f934e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-70d86995-8ecc-4a26-991f-34dc06f934e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:56.350265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:56.351156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:56.352091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 211/420] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:56.359356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e592ad-8210-432a-b609-a32bb7526ce1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:56.365194 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d86995-8ecc-4a26-991f-34dc06f934e3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:56.469933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d86995-8ecc-4a26-991f-34dc06f934e3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:56.477359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:56.492363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:56.506745 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d30be3-1788-4172-92a1-b87b09717ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:56.507570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d30be3-1788-4172-92a1-b87b09717ccb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:56.507852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d30be3-1788-4172-92a1-b87b09717ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:56.508132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d30be3-1788-4172-92a1-b87b09717ccb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:56.519710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d30be3-1788-4172-92a1-b87b09717ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-52d30be3-1788-4172-92a1-b87b09717ccb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:56.528731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:56.530895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:56.532904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 212/421] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:56.541503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d30be3-1788-4172-92a1-b87b09717ccb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:56.575629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e592ad-8210-432a-b609-a32bb7526ce1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:56.635231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c6d93-1e0a-465d-8d6f-319bba8f67bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:56.636095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c6d93-1e0a-465d-8d6f-319bba8f67bc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:56.636399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c6d93-1e0a-465d-8d6f-319bba8f67bc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:56.636676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c6d93-1e0a-465d-8d6f-319bba8f67bc None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:56.637428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c6d93-1e0a-465d-8d6f-319bba8f67bc None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:56.637730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c6d93-1e0a-465d-8d6f-319bba8f67bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:56.639105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c6d93-1e0a-465d-8d6f-319bba8f67bc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:56.661296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 210/422] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:56.670065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c6d93-1e0a-465d-8d6f-319bba8f67bc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:56.721758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d30be3-1788-4172-92a1-b87b09717ccb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:56.759123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474a62b-48ee-4a65-bec7-149edae91617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:56.759715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474a62b-48ee-4a65-bec7-149edae91617 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:56.759994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474a62b-48ee-4a65-bec7-149edae91617 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:56.760327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474a62b-48ee-4a65-bec7-149edae91617 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:56.761070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474a62b-48ee-4a65-bec7-149edae91617 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:56.761373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474a62b-48ee-4a65-bec7-149edae91617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:56.763173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474a62b-48ee-4a65-bec7-149edae91617 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:56.772001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 213/423] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:56.781190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474a62b-48ee-4a65-bec7-149edae91617 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:56.812503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:56.890415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474a62b-48ee-4a65-bec7-149edae91617 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:56.926618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac9808-4b45-4e42-9916-d24b5b1019bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:56.927537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac9808-4b45-4e42-9916-d24b5b1019bc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:56.927825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac9808-4b45-4e42-9916-d24b5b1019bc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:56.928098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac9808-4b45-4e42-9916-d24b5b1019bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:56.957630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c6d93-1e0a-465d-8d6f-319bba8f67bc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:57.003333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac9808-4b45-4e42-9916-d24b5b1019bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:57.004819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac9808-4b45-4e42-9916-d24b5b1019bc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:57.012670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:57.032994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f755-a8f3-4b77-8c8b-0ba1ceb77fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:57.033665 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f755-a8f3-4b77-8c8b-0ba1ceb77fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:57.033952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f755-a8f3-4b77-8c8b-0ba1ceb77fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:57.034232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f755-a8f3-4b77-8c8b-0ba1ceb77fe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:57.044747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:57.113245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 214/424] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:57.128063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac9808-4b45-4e42-9916-d24b5b1019bc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:57.155825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f755-a8f3-4b77-8c8b-0ba1ceb77fe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:57.157318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f755-a8f3-4b77-8c8b-0ba1ceb77fe7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:57.240204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac9808-4b45-4e42-9916-d24b5b1019bc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:57.284758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:57.298579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26d95f-8933-4ec7-9074-b5a5dbf28438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:57.299439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d95f-8933-4ec7-9074-b5a5dbf28438 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:57.299716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d95f-8933-4ec7-9074-b5a5dbf28438 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:57.299987 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d95f-8933-4ec7-9074-b5a5dbf28438 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:57.302664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26d95f-8933-4ec7-9074-b5a5dbf28438 None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:57.304113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26d95f-8933-4ec7-9074-b5a5dbf28438 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:57.306622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 215/425] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:57 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:57.313509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26d95f-8933-4ec7-9074-b5a5dbf28438 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:57.375286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 211/426] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:57.380470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f755-a8f3-4b77-8c8b-0ba1ceb77fe7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:57.433073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:57.548854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26d95f-8933-4ec7-9074-b5a5dbf28438 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:57.559898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f755-a8f3-4b77-8c8b-0ba1ceb77fe7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:57.586100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1168be-38cb-44f9-91df-ff580b771b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:57.587214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1168be-38cb-44f9-91df-ff580b771b9c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:57.587677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1168be-38cb-44f9-91df-ff580b771b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:57.588131 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1168be-38cb-44f9-91df-ff580b771b9c None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:57.589819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1168be-38cb-44f9-91df-ff580b771b9c None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:57.591477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1168be-38cb-44f9-91df-ff580b771b9c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:57.593892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 216/427] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:57 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:57.595337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766dd4d-f6f4-4b8c-8c34-fd1cd1d452c7 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:57.595827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766dd4d-f6f4-4b8c-8c34-fd1cd1d452c7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:57.596329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766dd4d-f6f4-4b8c-8c34-fd1cd1d452c7 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:57.602165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 217/428] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:57.603125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7bd64-93b4-4bb6-89ce-0bf9d2ecb566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:57.603477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766dd4d-f6f4-4b8c-8c34-fd1cd1d452c7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:57.604229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7bd64-93b4-4bb6-89ce-0bf9d2ecb566 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:57.605032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7bd64-93b4-4bb6-89ce-0bf9d2ecb566 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:57.605316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7bd64-93b4-4bb6-89ce-0bf9d2ecb566 None admin] PATH_INFO: `/v3/projects/380cd1f0793045d280c092a275d9666a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:57.606713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7bd64-93b4-4bb6-89ce-0bf9d2ecb566 None admin] RBAC: Authorizing `identity:get_project(project_id=380cd1f0793045d280c092a275d9666a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:57.608575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7bd64-93b4-4bb6-89ce-0bf9d2ecb566 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:57.614804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 212/429] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:57 2022] GET /identity/v3/projects/380cd1f0793045d280c092a275d9666a => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:57.614804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a8544-98db-4f21-948e-805c6bcbb6bc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:57.614804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a8544-98db-4f21-948e-805c6bcbb6bc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:57.614804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a8544-98db-4f21-948e-805c6bcbb6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:57.631405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c31a8544-98db-4f21-948e-805c6bcbb6bc None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:57.632626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c31a8544-98db-4f21-948e-805c6bcbb6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:57.690563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:57.717268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 213/430] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:57.718140 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a8544-98db-4f21-948e-805c6bcbb6bc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:57.733044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766dd4d-f6f4-4b8c-8c34-fd1cd1d452c7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:57.746986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:57.773604 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ae59-83d2-41f9-bbe9-a08999b62a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:57.774372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ae59-83d2-41f9-bbe9-a08999b62a3e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:57.774719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ae59-83d2-41f9-bbe9-a08999b62a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:57.775058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ae59-83d2-41f9-bbe9-a08999b62a3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:57.776090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7ae59-83d2-41f9-bbe9-a08999b62a3e None admin] RBAC: Adding query filter params (group.id=3794601340474cd89c8a29d6a289089e, role.id=110c20a9ff1e455d8c43ffdfbdd1447b, scope.project.id=380cd1f0793045d280c092a275d9666a) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:57.776489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7ae59-83d2-41f9-bbe9-a08999b62a3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:57.778099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7ae59-83d2-41f9-bbe9-a08999b62a3e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:57.784926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 218/431] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jul 21 17:20:57 2022] GET /identity/v3/role_assignments?group.id=3794601340474cd89c8a29d6a289089e&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=110c20a9ff1e455d8c43ffdfbdd1447b => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:57.785587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ae59-83d2-41f9-bbe9-a08999b62a3e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:57.897894 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ae59-83d2-41f9-bbe9-a08999b62a3e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:57.922367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a8544-98db-4f21-948e-805c6bcbb6bc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:57.936165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7afc44-b725-4c4d-9231-f4c40775170d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:57.937091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7afc44-b725-4c4d-9231-f4c40775170d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:57.937443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7afc44-b725-4c4d-9231-f4c40775170d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:57.937964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7afc44-b725-4c4d-9231-f4c40775170d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:57.950772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:57.951319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7afc44-b725-4c4d-9231-f4c40775170d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-3e7afc44-b725-4c4d-9231-f4c40775170d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:57.960924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:57.961889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:20:57.962679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:20:57.965633 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 219/432] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:20:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:57.971487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7afc44-b725-4c4d-9231-f4c40775170d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:58.028287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf7764-9503-405b-9050-e84269aad34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:58.030114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf7764-9503-405b-9050-e84269aad34f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.030114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf7764-9503-405b-9050-e84269aad34f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.030114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf7764-9503-405b-9050-e84269aad34f None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.031632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf7764-9503-405b-9050-e84269aad34f None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:58.033118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf7764-9503-405b-9050-e84269aad34f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:58.034905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 214/433] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:57 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:58.041087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf7764-9503-405b-9050-e84269aad34f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:58.119404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7afc44-b725-4c4d-9231-f4c40775170d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:58.147015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:58.158518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf7764-9503-405b-9050-e84269aad34f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:58.176543 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8e272-3e64-4b0d-af02-309cca74199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:58.177361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8e272-3e64-4b0d-af02-309cca74199a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.177793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8e272-3e64-4b0d-af02-309cca74199a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.178172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8e272-3e64-4b0d-af02-309cca74199a None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.179054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8e272-3e64-4b0d-af02-309cca74199a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:58.179469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8e272-3e64-4b0d-af02-309cca74199a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:58.181001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8e272-3e64-4b0d-af02-309cca74199a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:58.192495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 220/434] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:20:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:58.197012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55ba83-0c46-45a5-bc6c-993f9fb6e991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:58.197948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8e272-3e64-4b0d-af02-309cca74199a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:58.198216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ba83-0c46-45a5-bc6c-993f9fb6e991 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.198621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ba83-0c46-45a5-bc6c-993f9fb6e991 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.199028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ba83-0c46-45a5-bc6c-993f9fb6e991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.200252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55ba83-0c46-45a5-bc6c-993f9fb6e991 None admin] RBAC: Adding query filter params (group.id=3794601340474cd89c8a29d6a289089e, role.id=d2c2845de1d548bb9cfb46f70619b3f8, scope.project.id=380cd1f0793045d280c092a275d9666a) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:58.200775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55ba83-0c46-45a5-bc6c-993f9fb6e991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:58.202342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55ba83-0c46-45a5-bc6c-993f9fb6e991 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:58.209030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 215/435] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jul 21 17:20:58 2022] GET /identity/v3/role_assignments?group.id=3794601340474cd89c8a29d6a289089e&scope.project.id=380cd1f0793045d280c092a275d9666a&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:58.263381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:58.291277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6053f8-71a9-47a8-b58b-25b3ef493cae None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.292611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6053f8-71a9-47a8-b58b-25b3ef493cae None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.292984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6053f8-71a9-47a8-b58b-25b3ef493cae None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.298843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 216/436] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:20:58.300831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0e5d5-c055-4f8c-8830-b970fed3bac0 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.301129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0e5d5-c055-4f8c-8830-b970fed3bac0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.301498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0e5d5-c055-4f8c-8830-b970fed3bac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.320863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ceb0e5d5-c055-4f8c-8830-b970fed3bac0 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:58.322840 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ceb0e5d5-c055-4f8c-8830-b970fed3bac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:58.324048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8e272-3e64-4b0d-af02-309cca74199a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:58.369112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059db86-8458-4285-bf7d-e5bc3c83cf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:58.370011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059db86-8458-4285-bf7d-e5bc3c83cf22 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.370296 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059db86-8458-4285-bf7d-e5bc3c83cf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.370574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059db86-8458-4285-bf7d-e5bc3c83cf22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.412077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 217/437] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:20:58.439625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677e0ef-87fa-4f7a-b329-bffa68cf36d6 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.439912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677e0ef-87fa-4f7a-b329-bffa68cf36d6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.440468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677e0ef-87fa-4f7a-b329-bffa68cf36d6 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.442815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 218/438] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:20:58.446747 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677e0ef-87fa-4f7a-b329-bffa68cf36d6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:58.459726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059db86-8458-4285-bf7d-e5bc3c83cf22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:58.461214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059db86-8458-4285-bf7d-e5bc3c83cf22 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:58.522129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:58.553721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:58.576171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677e0ef-87fa-4f7a-b329-bffa68cf36d6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:58.586050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 221/439] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:58.587175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36492b-36f9-4196-b8df-1a140005b557 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.587496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36492b-36f9-4196-b8df-1a140005b557 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.587723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36492b-36f9-4196-b8df-1a140005b557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.606672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7f36492b-36f9-4196-b8df-1a140005b557 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:20:58.607899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7f36492b-36f9-4196-b8df-1a140005b557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:20:58.617705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90b98a-b18c-487f-9e4c-00a8688bcaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:58.618508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90b98a-b18c-487f-9e4c-00a8688bcaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.618791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90b98a-b18c-487f-9e4c-00a8688bcaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.619072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90b98a-b18c-487f-9e4c-00a8688bcaf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.631541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90b98a-b18c-487f-9e4c-00a8688bcaf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-dc90b98a-b18c-487f-9e4c-00a8688bcaf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:58.641037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:58.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:58.642751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 219/440] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:20:58.643560 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90b98a-b18c-487f-9e4c-00a8688bcaf2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:58.689140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 222/441] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:20:58.689906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36492b-36f9-4196-b8df-1a140005b557 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:58.782071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90b98a-b18c-487f-9e4c-00a8688bcaf2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:58.804712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:58.819186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6683f1-7e38-4b07-80ea-d6ef30f87fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:58.820054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6683f1-7e38-4b07-80ea-d6ef30f87fdb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.820365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6683f1-7e38-4b07-80ea-d6ef30f87fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.820646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6683f1-7e38-4b07-80ea-d6ef30f87fdb None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.821826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6683f1-7e38-4b07-80ea-d6ef30f87fdb None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:58.823907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6683f1-7e38-4b07-80ea-d6ef30f87fdb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:58.825908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 220/442] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:58 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:58.826130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36492b-36f9-4196-b8df-1a140005b557 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:58.826681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6683f1-7e38-4b07-80ea-d6ef30f87fdb None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:58.842508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:58.861730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e2ecf-105a-4c54-98bc-b339e3ee3770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:58.862396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e2ecf-105a-4c54-98bc-b339e3ee3770 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.862676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e2ecf-105a-4c54-98bc-b339e3ee3770 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.862949 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e2ecf-105a-4c54-98bc-b339e3ee3770 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.863687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e2ecf-105a-4c54-98bc-b339e3ee3770 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:58.863990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e2ecf-105a-4c54-98bc-b339e3ee3770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:58.865379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e2ecf-105a-4c54-98bc-b339e3ee3770 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:58.873984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 223/443] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:20:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:58.874763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e2ecf-105a-4c54-98bc-b339e3ee3770 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:58.927835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6683f1-7e38-4b07-80ea-d6ef30f87fdb None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:58.963734 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660614fe-7923-425b-85fa-ca6b2ab2aafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:58.964485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660614fe-7923-425b-85fa-ca6b2ab2aafa None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:58.964770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660614fe-7923-425b-85fa-ca6b2ab2aafa None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:58.965048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660614fe-7923-425b-85fa-ca6b2ab2aafa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:58.974949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660614fe-7923-425b-85fa-ca6b2ab2aafa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:20:58.981086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e2ecf-105a-4c54-98bc-b339e3ee3770 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-660614fe-7923-425b-85fa-ca6b2ab2aafa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:20:58.982671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:20:58.983719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:20:58.984571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 221/444] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:20:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:20:58.985243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660614fe-7923-425b-85fa-ca6b2ab2aafa None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:59.017623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628a23e-1ce3-47a8-9f77-352a024b8854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:59.018461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628a23e-1ce3-47a8-9f77-352a024b8854 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:59.018741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628a23e-1ce3-47a8-9f77-352a024b8854 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:59.019015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628a23e-1ce3-47a8-9f77-352a024b8854 None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:59.020169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628a23e-1ce3-47a8-9f77-352a024b8854 None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:59.021651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628a23e-1ce3-47a8-9f77-352a024b8854 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:59.023411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 224/445] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:58 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:59.024061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628a23e-1ce3-47a8-9f77-352a024b8854 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:59.037015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:59.060792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:59.086339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660614fe-7923-425b-85fa-ca6b2ab2aafa None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:59.118675 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892232a9-cc36-409e-b189-301c3d5d4c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:59.119323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892232a9-cc36-409e-b189-301c3d5d4c12 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:59.120147 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892232a9-cc36-409e-b189-301c3d5d4c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:59.120457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892232a9-cc36-409e-b189-301c3d5d4c12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:59.125305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628a23e-1ce3-47a8-9f77-352a024b8854 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:59.158309 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c815582-e128-420d-9917-e4c3a4642376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:59.158963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c815582-e128-420d-9917-e4c3a4642376 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:59.159247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c815582-e128-420d-9917-e4c3a4642376 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:59.159517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c815582-e128-420d-9917-e4c3a4642376 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:59.160262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c815582-e128-420d-9917-e4c3a4642376 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:59.160599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c815582-e128-420d-9917-e4c3a4642376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:59.161974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c815582-e128-420d-9917-e4c3a4642376 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:59.170856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 225/446] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:20:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:59.171542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c815582-e128-420d-9917-e4c3a4642376 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:59.190084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892232a9-cc36-409e-b189-301c3d5d4c12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:59.191557 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892232a9-cc36-409e-b189-301c3d5d4c12 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:59.241707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:59.267303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:59.272392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c815582-e128-420d-9917-e4c3a4642376 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:59.293531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 222/447] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:59.294370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892232a9-cc36-409e-b189-301c3d5d4c12 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:59.310533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d2cae-6319-4f12-92e7-ebfa5802627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:59.311188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d2cae-6319-4f12-92e7-ebfa5802627a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:59.311473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d2cae-6319-4f12-92e7-ebfa5802627a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:59.311755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d2cae-6319-4f12-92e7-ebfa5802627a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:59.312618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d2cae-6319-4f12-92e7-ebfa5802627a None admin] RBAC: Adding query filter params (group.id=3794601340474cd89c8a29d6a289089e, role.id=110c20a9ff1e455d8c43ffdfbdd1447b, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:59.312942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d2cae-6319-4f12-92e7-ebfa5802627a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:59.314380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d2cae-6319-4f12-92e7-ebfa5802627a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:59.320367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 226/448] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jul 21 17:20:59 2022] GET /identity/v3/role_assignments?group.id=3794601340474cd89c8a29d6a289089e&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=110c20a9ff1e455d8c43ffdfbdd1447b => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:59.321206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d2cae-6319-4f12-92e7-ebfa5802627a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:59.405584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892232a9-cc36-409e-b189-301c3d5d4c12 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:59.440666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d2cae-6319-4f12-92e7-ebfa5802627a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:59.449495 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554388ef-79f7-4aee-811d-969c04b3e007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:59.450979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554388ef-79f7-4aee-811d-969c04b3e007 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:59.451265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554388ef-79f7-4aee-811d-969c04b3e007 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:59.451545 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554388ef-79f7-4aee-811d-969c04b3e007 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:59.480950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5269d9a-5fe5-4972-b27c-ec3f1838d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:59.482591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5269d9a-5fe5-4972-b27c-ec3f1838d74e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:59.482591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5269d9a-5fe5-4972-b27c-ec3f1838d74e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:59.482591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5269d9a-5fe5-4972-b27c-ec3f1838d74e None admin] PATH_INFO: `/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:59.483741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5269d9a-5fe5-4972-b27c-ec3f1838d74e None admin] RBAC: Authorizing `identity:get_project(project_id=4819e5a17eb24efcb8ccbb51a48b30ca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:59.485542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5269d9a-5fe5-4972-b27c-ec3f1838d74e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:59.487055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 227/449] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:59 2022] GET /identity/v3/projects/4819e5a17eb24efcb8ccbb51a48b30ca => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:59.492581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5269d9a-5fe5-4972-b27c-ec3f1838d74e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:59.497403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:59.509264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:59.539582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554388ef-79f7-4aee-811d-969c04b3e007 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:59.541089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554388ef-79f7-4aee-811d-969c04b3e007 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:59.599893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5269d9a-5fe5-4972-b27c-ec3f1838d74e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:59.633238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0e85d-02ec-4d3b-9fae-7e1b2fc6f847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:59.634096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0e85d-02ec-4d3b-9fae-7e1b2fc6f847 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:59.634382 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0e85d-02ec-4d3b-9fae-7e1b2fc6f847 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:59.634657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0e85d-02ec-4d3b-9fae-7e1b2fc6f847 None admin] PATH_INFO: `/v3/groups/3794601340474cd89c8a29d6a289089e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:59.635841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0e85d-02ec-4d3b-9fae-7e1b2fc6f847 None admin] RBAC: Authorizing `identity:get_group(group_id=3794601340474cd89c8a29d6a289089e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:59.637312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0e85d-02ec-4d3b-9fae-7e1b2fc6f847 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:59.638927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 228/450] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:59 2022] GET /identity/v3/groups/3794601340474cd89c8a29d6a289089e => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:59.642162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 223/451] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:20:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:20:59.644614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0e85d-02ec-4d3b-9fae-7e1b2fc6f847 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:59.647295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554388ef-79f7-4aee-811d-969c04b3e007 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:59.722019 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:59.722723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:20:59.749105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554388ef-79f7-4aee-811d-969c04b3e007 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:59.753619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0e85d-02ec-4d3b-9fae-7e1b2fc6f847 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:59.786093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea35459-8eee-453e-b466-9fc98f211cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:59.786958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea35459-8eee-453e-b466-9fc98f211cbd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:59.787240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea35459-8eee-453e-b466-9fc98f211cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:59.787404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5512a-189d-485c-afb7-b82815cc1c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:59.787515 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea35459-8eee-453e-b466-9fc98f211cbd None admin] PATH_INFO: `/v3/projects/ee47bf64e42844579243da3bda418cc7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:59.788057 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5512a-189d-485c-afb7-b82815cc1c7a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:59.788369 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5512a-189d-485c-afb7-b82815cc1c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:59.788650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5512a-189d-485c-afb7-b82815cc1c7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:59.788815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea35459-8eee-453e-b466-9fc98f211cbd None admin] RBAC: Authorizing `identity:get_project(project_id=ee47bf64e42844579243da3bda418cc7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:59.789476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5512a-189d-485c-afb7-b82815cc1c7a None admin] RBAC: Adding query filter params (group.id=3794601340474cd89c8a29d6a289089e, role.id=d2c2845de1d548bb9cfb46f70619b3f8, scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:20:59.789790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5512a-189d-485c-afb7-b82815cc1c7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:59.790253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea35459-8eee-453e-b466-9fc98f211cbd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:59.791214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5512a-189d-485c-afb7-b82815cc1c7a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:59.797790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 229/452] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jul 21 17:20:59 2022] GET /identity/v3/role_assignments?group.id=3794601340474cd89c8a29d6a289089e&scope.project.id=4819e5a17eb24efcb8ccbb51a48b30ca&role.id=d2c2845de1d548bb9cfb46f70619b3f8 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:59.804460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 224/453] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:20:59 2022] GET /identity/v3/projects/ee47bf64e42844579243da3bda418cc7 => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:20:59.805785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5512a-189d-485c-afb7-b82815cc1c7a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:20:59.926679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5512a-189d-485c-afb7-b82815cc1c7a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:20:59.962513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ce1d0-f89e-49b4-b4fc-8009ef717da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:20:59.963983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ce1d0-f89e-49b4-b4fc-8009ef717da8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:20:59.964275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ce1d0-f89e-49b4-b4fc-8009ef717da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:20:59.964581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ce1d0-f89e-49b4-b4fc-8009ef717da8 None admin] PATH_INFO: `/v3/groups/a91f682189014c91a1fd5df1ab74f15d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:20:59.965771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ce1d0-f89e-49b4-b4fc-8009ef717da8 None admin] RBAC: Authorizing `identity:get_group(group_id=a91f682189014c91a1fd5df1ab74f15d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:20:59.967234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ce1d0-f89e-49b4-b4fc-8009ef717da8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:20:59.970561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 230/454] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Jul 21 17:20:59 2022] GET /identity/v3/groups/a91f682189014c91a1fd5df1ab74f15d => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:20:59.974905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea35459-8eee-453e-b466-9fc98f211cbd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:00.079575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea35459-8eee-453e-b466-9fc98f211cbd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:00.114242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0130a-18ce-4168-8cc8-931e49e8d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:00.114893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0130a-18ce-4168-8cc8-931e49e8d855 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:00.115172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0130a-18ce-4168-8cc8-931e49e8d855 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:00.115440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0130a-18ce-4168-8cc8-931e49e8d855 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:00.116266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0130a-18ce-4168-8cc8-931e49e8d855 None admin] RBAC: Adding query filter params (group.id=a91f682189014c91a1fd5df1ab74f15d, role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=ee47bf64e42844579243da3bda418cc7) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:00.116596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0130a-18ce-4168-8cc8-931e49e8d855 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:00.118025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0130a-18ce-4168-8cc8-931e49e8d855 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:00.123987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 225/455] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Jul 21 17:20:59 2022] GET /identity/v3/role_assignments?group.id=a91f682189014c91a1fd5df1ab74f15d&scope.project.id=ee47bf64e42844579243da3bda418cc7&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:02.878957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35528fa-27b1-41a6-ae2e-5e81e920c64b None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:02.878957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35528fa-27b1-41a6-ae2e-5e81e920c64b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:02.878957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35528fa-27b1-41a6-ae2e-5e81e920c64b None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:02.879905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 231/456] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:02.884098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf075-7204-46ef-9e9f-eb576333fbd0 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:02.884529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf075-7204-46ef-9e9f-eb576333fbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:02.884852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf075-7204-46ef-9e9f-eb576333fbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:02.901872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:02.917876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ce8cf075-7204-46ef-9e9f-eb576333fbd0 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:02.919198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ce8cf075-7204-46ef-9e9f-eb576333fbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:02.972993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a89e7-d296-4058-981f-7f2a48b65fa8 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:02.973605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a89e7-d296-4058-981f-7f2a48b65fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:02.974105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a89e7-d296-4058-981f-7f2a48b65fa8 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:02.976288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 232/457] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:02.982332 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aecb8f-e772-40dd-9848-400a28be5cb9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:02.982883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aecb8f-e772-40dd-9848-400a28be5cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:02.983357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aecb8f-e772-40dd-9848-400a28be5cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:02.996644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:03.012352 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e7aecb8f-e772-40dd-9848-400a28be5cb9 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:03.013555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e7aecb8f-e772-40dd-9848-400a28be5cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:03.029243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 226/458] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:03.087783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cf075-7204-46ef-9e9f-eb576333fbd0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:03.115704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 233/459] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:21:03.198531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aecb8f-e772-40dd-9848-400a28be5cb9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:03.259698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cf075-7204-46ef-9e9f-eb576333fbd0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:03.301974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ddae2-6feb-428f-961a-1b8b9a257e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:03.302784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ddae2-6feb-428f-961a-1b8b9a257e24 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.303311 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ddae2-6feb-428f-961a-1b8b9a257e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.303596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ddae2-6feb-428f-961a-1b8b9a257e24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.307881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:03.318993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ddae2-6feb-428f-961a-1b8b9a257e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:03.321181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ddae2-6feb-428f-961a-1b8b9a257e24 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-452ddae2-6feb-428f-961a-1b8b9a257e24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:03.346951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:03.347874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:03.348768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:03.348768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:03.348768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:03.348768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:03.349804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 227/460] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:03.350564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1a02a-4baa-4e10-8d6e-5323ceec550a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.350843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1a02a-4baa-4e10-8d6e-5323ceec550a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.351115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1a02a-4baa-4e10-8d6e-5323ceec550a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.356384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 228/461] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:03.357602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1a02a-4baa-4e10-8d6e-5323ceec550a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:03.372012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aecb8f-e772-40dd-9848-400a28be5cb9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:03.412939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fe3ce-4c9f-47cc-9019-f4d8a352b025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:03.413729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fe3ce-4c9f-47cc-9019-f4d8a352b025 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.414010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fe3ce-4c9f-47cc-9019-f4d8a352b025 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.414395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fe3ce-4c9f-47cc-9019-f4d8a352b025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.416511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:03.425153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fe3ce-4c9f-47cc-9019-f4d8a352b025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:03.426607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fe3ce-4c9f-47cc-9019-f4d8a352b025 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-fe2fe3ce-4c9f-47cc-9019-f4d8a352b025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:03.445174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:03.446073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:03.453252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:03.453252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:03.453252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:03.453252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:03.453252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 234/462] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:03.453252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87d12-dcb9-4f14-acf4-b702aca1a1b1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.453252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87d12-dcb9-4f14-acf4-b702aca1a1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.453252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87d12-dcb9-4f14-acf4-b702aca1a1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.468718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-edb87d12-dcb9-4f14-acf4-b702aca1a1b1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:03.470072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-edb87d12-dcb9-4f14-acf4-b702aca1a1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:03.479910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1a02a-4baa-4e10-8d6e-5323ceec550a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:03.515196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1810d3-ff63-4768-8e0c-feda9bd8047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:03.515863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1810d3-ff63-4768-8e0c-feda9bd8047f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.516141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1810d3-ff63-4768-8e0c-feda9bd8047f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.516452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1810d3-ff63-4768-8e0c-feda9bd8047f None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.517182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1810d3-ff63-4768-8e0c-feda9bd8047f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:03.517484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1810d3-ff63-4768-8e0c-feda9bd8047f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:03.518846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1810d3-ff63-4768-8e0c-feda9bd8047f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:03.531180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 229/463] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:03.534181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5425d-28bf-491e-8a1b-243dda9883e9 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.535032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5425d-28bf-491e-8a1b-243dda9883e9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.535476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5425d-28bf-491e-8a1b-243dda9883e9 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.537178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 230/464] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:03.541530 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5425d-28bf-491e-8a1b-243dda9883e9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:03.542947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 235/465] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:03.548341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb87d12-dcb9-4f14-acf4-b702aca1a1b1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:03.597826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:03.645705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb87d12-dcb9-4f14-acf4-b702aca1a1b1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:03.652978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5425d-28bf-491e-8a1b-243dda9883e9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:03.662856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:03.683732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a41905-15fb-465a-8e75-18f3b420864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:03.684496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a41905-15fb-465a-8e75-18f3b420864b None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.684704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a41905-15fb-465a-8e75-18f3b420864b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.684980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a41905-15fb-465a-8e75-18f3b420864b None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.685674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a41905-15fb-465a-8e75-18f3b420864b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:03.687115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a41905-15fb-465a-8e75-18f3b420864b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:03.691120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890eddcd-7c01-414b-aa5e-7ea7f0c33115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:03.691797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890eddcd-7c01-414b-aa5e-7ea7f0c33115 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.692076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890eddcd-7c01-414b-aa5e-7ea7f0c33115 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.692377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890eddcd-7c01-414b-aa5e-7ea7f0c33115 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.693114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890eddcd-7c01-414b-aa5e-7ea7f0c33115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:03.693421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890eddcd-7c01-414b-aa5e-7ea7f0c33115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:03.695021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890eddcd-7c01-414b-aa5e-7ea7f0c33115 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:03.708632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 236/466] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:21:03 2022] POST /identity/v3/users => generated 248 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:03.713489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 231/467] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:03.713489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890eddcd-7c01-414b-aa5e-7ea7f0c33115 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:03.717870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc968ac9-16e3-4183-84e5-579e732e3f9b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.718159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc968ac9-16e3-4183-84e5-579e732e3f9b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.718441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc968ac9-16e3-4183-84e5-579e732e3f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.746095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-cc968ac9-16e3-4183-84e5-579e732e3f9b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:03.748823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cc968ac9-16e3-4183-84e5-579e732e3f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:03.843603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890eddcd-7c01-414b-aa5e-7ea7f0c33115 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:03.856038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:03.857128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 237/468] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:03.857817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc968ac9-16e3-4183-84e5-579e732e3f9b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:03.891392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf941c4d-3812-4b8c-ae6d-6ca2b3cd5da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:03.892198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf941c4d-3812-4b8c-ae6d-6ca2b3cd5da3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.892547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf941c4d-3812-4b8c-ae6d-6ca2b3cd5da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.892819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf941c4d-3812-4b8c-ae6d-6ca2b3cd5da3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.903096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf941c4d-3812-4b8c-ae6d-6ca2b3cd5da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:03.904587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf941c4d-3812-4b8c-ae6d-6ca2b3cd5da3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-cf941c4d-3812-4b8c-ae6d-6ca2b3cd5da3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:03.924099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:03.925068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:03.925896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:03.925896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:03.925896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:03.925896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:03.925896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 232/469] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:03.926948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f7b3c-2075-4a08-9e37-4f1103fd9f2c None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:03.927229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f7b3c-2075-4a08-9e37-4f1103fd9f2c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:03.927499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f7b3c-2075-4a08-9e37-4f1103fd9f2c None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:03.929055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 233/470] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:03.929743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f7b3c-2075-4a08-9e37-4f1103fd9f2c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:03.990079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc968ac9-16e3-4183-84e5-579e732e3f9b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:03.991547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:04.028534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef05f2-dfb1-4271-bb11-7d3f711d14f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:04.029542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef05f2-dfb1-4271-bb11-7d3f711d14f7 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:04.030005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef05f2-dfb1-4271-bb11-7d3f711d14f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:04.030557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef05f2-dfb1-4271-bb11-7d3f711d14f7 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:04.031545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef05f2-dfb1-4271-bb11-7d3f711d14f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:04.033187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef05f2-dfb1-4271-bb11-7d3f711d14f7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:04.048533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f7b3c-2075-4a08-9e37-4f1103fd9f2c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:04.055194 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 238/471] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:21:03 2022] POST /identity/v3/users => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:04.056454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef05f2-dfb1-4271-bb11-7d3f711d14f7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:04.099259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd798921-8ae3-4b55-800c-0f3b448ce554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:04.100175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd798921-8ae3-4b55-800c-0f3b448ce554 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:04.100556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd798921-8ae3-4b55-800c-0f3b448ce554 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:04.100912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd798921-8ae3-4b55-800c-0f3b448ce554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:04.114509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd798921-8ae3-4b55-800c-0f3b448ce554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:04.116533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd798921-8ae3-4b55-800c-0f3b448ce554 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-fd798921-8ae3-4b55-800c-0f3b448ce554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:04.143191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:04.144317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:04.145193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:04.145193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:04.145193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:04.145193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:04.145898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 234/472] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:04.147869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e15f9-0b37-4ff6-8214-11de26805d26 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:04.148158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e15f9-0b37-4ff6-8214-11de26805d26 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:04.148462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e15f9-0b37-4ff6-8214-11de26805d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:04.170462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d43e15f9-0b37-4ff6-8214-11de26805d26 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:04.172055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d43e15f9-0b37-4ff6-8214-11de26805d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:04.182876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:04.195851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef05f2-dfb1-4271-bb11-7d3f711d14f7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:04.237635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0e18b-6bc0-4d1f-a487-8f9d954849c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:04.238307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0e18b-6bc0-4d1f-a487-8f9d954849c1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:04.238591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0e18b-6bc0-4d1f-a487-8f9d954849c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:04.238864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0e18b-6bc0-4d1f-a487-8f9d954849c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:04.239681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0e18b-6bc0-4d1f-a487-8f9d954849c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:04.239974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0e18b-6bc0-4d1f-a487-8f9d954849c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:04.242248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0e18b-6bc0-4d1f-a487-8f9d954849c1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:04.254810 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 239/473] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:04.255482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0e18b-6bc0-4d1f-a487-8f9d954849c1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:04.267474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 235/474] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:04.268644 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e15f9-0b37-4ff6-8214-11de26805d26 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:04.284001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:04.387215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0e18b-6bc0-4d1f-a487-8f9d954849c1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:04.397015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e15f9-0b37-4ff6-8214-11de26805d26 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:04.432484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c97dc5-0c5b-4c6b-82c6-7c7576477a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:04.433702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c97dc5-0c5b-4c6b-82c6-7c7576477a06 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:04.434501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c97dc5-0c5b-4c6b-82c6-7c7576477a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:04.435254 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c97dc5-0c5b-4c6b-82c6-7c7576477a06 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:04.436275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c97dc5-0c5b-4c6b-82c6-7c7576477a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:04.436811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c97dc5-0c5b-4c6b-82c6-7c7576477a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:04.438717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c97dc5-0c5b-4c6b-82c6-7c7576477a06 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:04.438978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7d427-cc5e-4955-8017-3e7586c008aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:04.439660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7d427-cc5e-4955-8017-3e7586c008aa None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:04.439943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7d427-cc5e-4955-8017-3e7586c008aa None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:04.440222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7d427-cc5e-4955-8017-3e7586c008aa None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:04.440973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7d427-cc5e-4955-8017-3e7586c008aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:04.442713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7d427-cc5e-4955-8017-3e7586c008aa None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:04.456033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 240/475] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:04.458080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c97dc5-0c5b-4c6b-82c6-7c7576477a06 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:04.466652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:04.474085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 236/476] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:21:04 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:04.480701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7d427-cc5e-4955-8017-3e7586c008aa None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:04.558580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:04.583425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c97dc5-0c5b-4c6b-82c6-7c7576477a06 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:04.646434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7d427-cc5e-4955-8017-3e7586c008aa None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:04.650739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544fd7d-22b1-4bca-ad2f-41886ccad981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:04.651687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544fd7d-22b1-4bca-ad2f-41886ccad981 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:04.652170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544fd7d-22b1-4bca-ad2f-41886ccad981 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:04.652670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544fd7d-22b1-4bca-ad2f-41886ccad981 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:04.653653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544fd7d-22b1-4bca-ad2f-41886ccad981 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:04.655367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544fd7d-22b1-4bca-ad2f-41886ccad981 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:04.669030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.sql.core [None req-f544fd7d-22b1-4bca-ad2f-41886ccad981 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 17:21:04.669030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 17:21:04.669030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [parameters: {'id': '1d3c0ca0acc44d00852836a3f1840366', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 17:21:04.669030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80210) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 17:21:04.687834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651816-8c58-46e9-a7c2-8e7e85c09dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:04.688644 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651816-8c58-46e9-a7c2-8e7e85c09dda None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:04.688983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651816-8c58-46e9-a7c2-8e7e85c09dda None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:04.689267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651816-8c58-46e9-a7c2-8e7e85c09dda None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:04.689985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651816-8c58-46e9-a7c2-8e7e85c09dda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:04.691543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651816-8c58-46e9-a7c2-8e7e85c09dda None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f544fd7d-22b1-4bca-ad2f-41886ccad981 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application conn.query(q) Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application result.read() Jul 21 17:21:04.701242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 17:21:04.702171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application next(self.gen) Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application yield resource Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application next(self.gen) Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise self.value Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application session.commit() Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 17:21:04.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self.session.flush() Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application compat.raise_( Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 17:21:04.704018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 17:21:04.704922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application conn.query(q) Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application result.read() Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 17:21:04.707627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [parameters: {'id': '1d3c0ca0acc44d00852836a3f1840366', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:04.713275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 17:21:04.715411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:04.717401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 241/477] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Jul 21 17:21:04 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:04.717401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544fd7d-22b1-4bca-ad2f-41886ccad981 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:04.739790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 237/478] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:21:04 2022] POST /identity/v3/users => generated 249 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:04.863351 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544fd7d-22b1-4bca-ad2f-41886ccad981 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:04.901324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c24f38-e144-49ce-9fc6-cd03104abe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:04.902168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c24f38-e144-49ce-9fc6-cd03104abe2c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:04.902451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c24f38-e144-49ce-9fc6-cd03104abe2c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:04.902728 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c24f38-e144-49ce-9fc6-cd03104abe2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:04.924325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c24f38-e144-49ce-9fc6-cd03104abe2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-18c24f38-e144-49ce-9fc6-cd03104abe2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:04.926132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:04.927006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:04.927813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 242/479] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:21:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:04.935514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651816-8c58-46e9-a7c2-8e7e85c09dda None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:05.002614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:05.051543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651816-8c58-46e9-a7c2-8e7e85c09dda None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:05.087505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc3601-218d-4cbc-9bbc-eddc3647d77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:05.088382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc3601-218d-4cbc-9bbc-eddc3647d77a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:05.088676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc3601-218d-4cbc-9bbc-eddc3647d77a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:05.088956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc3601-218d-4cbc-9bbc-eddc3647d77a None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:05.089712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc3601-218d-4cbc-9bbc-eddc3647d77a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:05.090020 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc3601-218d-4cbc-9bbc-eddc3647d77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:05.091403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc3601-218d-4cbc-9bbc-eddc3647d77a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:05.100268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 238/480] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:05.920241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f32c23-ad66-41f7-ae6d-1800b9aed792 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:05.920771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f32c23-ad66-41f7-ae6d-1800b9aed792 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:05.920868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f32c23-ad66-41f7-ae6d-1800b9aed792 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:05.922314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 243/481] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:05.927594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0c076-a4df-4859-8ec2-3c42da5f19e4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:05.927887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0c076-a4df-4859-8ec2-3c42da5f19e4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:05.928167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0c076-a4df-4859-8ec2-3c42da5f19e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:05.954608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fed0c076-a4df-4859-8ec2-3c42da5f19e4 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:05.955809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fed0c076-a4df-4859-8ec2-3c42da5f19e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:06.050505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 239/482] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:06.085663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f32c23-ad66-41f7-ae6d-1800b9aed792 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:06.093466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:06.243052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f32c23-ad66-41f7-ae6d-1800b9aed792 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:06.289085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0591fb-dce7-4de1-a96c-740b15bcb055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:06.289884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0591fb-dce7-4de1-a96c-740b15bcb055 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:06.290164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0591fb-dce7-4de1-a96c-740b15bcb055 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:06.290440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0591fb-dce7-4de1-a96c-740b15bcb055 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:06.302889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0591fb-dce7-4de1-a96c-740b15bcb055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-0a0591fb-dce7-4de1-a96c-740b15bcb055 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:06.316153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:06.318245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:06.320127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 244/483] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:06.329402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0c076-a4df-4859-8ec2-3c42da5f19e4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:06.399319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:06.471115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0c076-a4df-4859-8ec2-3c42da5f19e4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:06.477043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37b4fa-d54f-42aa-951a-6ee8acaf1dbe None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:06.477560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37b4fa-d54f-42aa-951a-6ee8acaf1dbe None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:06.478159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37b4fa-d54f-42aa-951a-6ee8acaf1dbe None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:06.480919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 245/484] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:06.499437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bb25-142f-4ccb-bdfc-cb3fe9ac70de None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:06.500460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bb25-142f-4ccb-bdfc-cb3fe9ac70de None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:06.501221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bb25-142f-4ccb-bdfc-cb3fe9ac70de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:06.515309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c168346-a2c2-4878-9a23-39c5e5a633c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:06.515857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c168346-a2c2-4878-9a23-39c5e5a633c5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:06.516138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c168346-a2c2-4878-9a23-39c5e5a633c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:06.520236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c168346-a2c2-4878-9a23-39c5e5a633c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:06.520236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c168346-a2c2-4878-9a23-39c5e5a633c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:06.520236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c168346-a2c2-4878-9a23-39c5e5a633c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:06.520236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c168346-a2c2-4878-9a23-39c5e5a633c5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:06.534859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3d10bb25-142f-4ccb-bdfc-cb3fe9ac70de None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:06.536073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3d10bb25-142f-4ccb-bdfc-cb3fe9ac70de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:06.550656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 240/485] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:06.555669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c168346-a2c2-4878-9a23-39c5e5a633c5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:06.614263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:06.629613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 246/486] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:06.669385 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10bb25-142f-4ccb-bdfc-cb3fe9ac70de None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:06.726223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c168346-a2c2-4878-9a23-39c5e5a633c5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:06.748625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:06.805134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9afc1-342c-479c-bd0f-1e4919e15d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:06.805873 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9afc1-342c-479c-bd0f-1e4919e15d5d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:06.805991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10bb25-142f-4ccb-bdfc-cb3fe9ac70de None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:06.806161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9afc1-342c-479c-bd0f-1e4919e15d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:06.806423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9afc1-342c-479c-bd0f-1e4919e15d5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:06.826818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9afc1-342c-479c-bd0f-1e4919e15d5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:06.828268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9afc1-342c-479c-bd0f-1e4919e15d5d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:06.842596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20aab6-ddcb-4baf-a132-d9cb997055c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:06.843400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20aab6-ddcb-4baf-a132-d9cb997055c8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:06.843687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20aab6-ddcb-4baf-a132-d9cb997055c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:06.844007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20aab6-ddcb-4baf-a132-d9cb997055c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:06.855216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20aab6-ddcb-4baf-a132-d9cb997055c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-0cb9afc1-342c-479c-bd0f-1e4919e15d5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:06.857145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:06.857944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:06.858769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:06.858769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:06.858769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:06.858769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-1c20aab6-ddcb-4baf-a132-d9cb997055c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:06.863047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:06.863900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:06.864733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:06.866012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 241/488] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:06.866151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 247/488] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:06.873304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9afc1-342c-479c-bd0f-1e4919e15d5d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:06.874578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20aab6-ddcb-4baf-a132-d9cb997055c8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:06.976944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:06.997068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9afc1-342c-479c-bd0f-1e4919e15d5d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:07.040562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20aab6-ddcb-4baf-a132-d9cb997055c8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:07.073087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf4885-a4f6-4b80-9680-ec1af22046e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:07.073694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf4885-a4f6-4b80-9680-ec1af22046e9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.073979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf4885-a4f6-4b80-9680-ec1af22046e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.074524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf4885-a4f6-4b80-9680-ec1af22046e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.075279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf4885-a4f6-4b80-9680-ec1af22046e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:07.075585 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf4885-a4f6-4b80-9680-ec1af22046e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:07.076982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf4885-a4f6-4b80-9680-ec1af22046e9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:07.077729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836b40-6d7d-476a-bfad-9bb390917a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:07.078654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836b40-6d7d-476a-bfad-9bb390917a8f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.079080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836b40-6d7d-476a-bfad-9bb390917a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.079540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836b40-6d7d-476a-bfad-9bb390917a8f None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.080419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2836b40-6d7d-476a-bfad-9bb390917a8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:07.080862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2836b40-6d7d-476a-bfad-9bb390917a8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:07.082473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2836b40-6d7d-476a-bfad-9bb390917a8f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:07.095506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 248/490] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:07.095677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 242/490] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:07.101789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf4885-a4f6-4b80-9680-ec1af22046e9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:07.101908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836b40-6d7d-476a-bfad-9bb390917a8f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:07.162559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:07.222259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836b40-6d7d-476a-bfad-9bb390917a8f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:07.258415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:07.285376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf4885-a4f6-4b80-9680-ec1af22046e9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:07.310391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3f1fb-2b3c-416d-8be7-743d9a022eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:07.311199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3f1fb-2b3c-416d-8be7-743d9a022eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.311481 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3f1fb-2b3c-416d-8be7-743d9a022eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.311754 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3f1fb-2b3c-416d-8be7-743d9a022eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.331212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f13287-0107-4616-ab16-2c2912a85f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:07.332405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f13287-0107-4616-ab16-2c2912a85f2a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.332943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f13287-0107-4616-ab16-2c2912a85f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.334399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f13287-0107-4616-ab16-2c2912a85f2a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.336226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3f1fb-2b3c-416d-8be7-743d9a022eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:07.346498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3f1fb-2b3c-416d-8be7-743d9a022eb0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:07.350649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f13287-0107-4616-ab16-2c2912a85f2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:07.357969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f13287-0107-4616-ab16-2c2912a85f2a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-16e3f1fb-2b3c-416d-8be7-743d9a022eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:07.369067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:07.370553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:07.371423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:07.371423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:07.371423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:07.371423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:07.373400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 249/491] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:07.379122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96e47e-6f3d-45ef-8693-0dcae43439f9 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.379633 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96e47e-6f3d-45ef-8693-0dcae43439f9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.380068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96e47e-6f3d-45ef-8693-0dcae43439f9 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.381622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 250/492] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f7f13287-0107-4616-ab16-2c2912a85f2a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:07.381922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:07.382949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:07.383854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:07.383854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:07.383854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:07.383854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:07.383854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:07.383854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:07.383854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:07.383854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:07.383854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 17:21:07.383854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:07.384453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 243/493] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:07.386545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96e47e-6f3d-45ef-8693-0dcae43439f9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:07.389913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14afecc-ee8d-4f6c-b6ae-a9db18f11307 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.390270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14afecc-ee8d-4f6c-b6ae-a9db18f11307 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.390615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14afecc-ee8d-4f6c-b6ae-a9db18f11307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.408934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e14afecc-ee8d-4f6c-b6ae-a9db18f11307 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:07.410213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e14afecc-ee8d-4f6c-b6ae-a9db18f11307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:07.497191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 244/494] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:07.498386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14afecc-ee8d-4f6c-b6ae-a9db18f11307 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:07.531116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:07.531390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96e47e-6f3d-45ef-8693-0dcae43439f9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:07.568287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e145a7e-d08c-47bb-8974-bba337443996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:07.568982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e145a7e-d08c-47bb-8974-bba337443996 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.569252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e145a7e-d08c-47bb-8974-bba337443996 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.569540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e145a7e-d08c-47bb-8974-bba337443996 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.570241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e145a7e-d08c-47bb-8974-bba337443996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:07.570545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e145a7e-d08c-47bb-8974-bba337443996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:07.571928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e145a7e-d08c-47bb-8974-bba337443996 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:07.590146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 251/495] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:07.590146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e145a7e-d08c-47bb-8974-bba337443996 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:07.617540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:07.624783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14afecc-ee8d-4f6c-b6ae-a9db18f11307 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:07.670015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9221431-a36a-48a0-b7c8-901b9d83afbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:07.670720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9221431-a36a-48a0-b7c8-901b9d83afbf None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.671004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9221431-a36a-48a0-b7c8-901b9d83afbf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.671279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9221431-a36a-48a0-b7c8-901b9d83afbf None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.672120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9221431-a36a-48a0-b7c8-901b9d83afbf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:07.672456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9221431-a36a-48a0-b7c8-901b9d83afbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:07.673900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9221431-a36a-48a0-b7c8-901b9d83afbf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:07.717236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e145a7e-d08c-47bb-8974-bba337443996 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:07.720485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 245/496] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jul 21 17:21:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:07.720485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9221431-a36a-48a0-b7c8-901b9d83afbf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:07.754493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27fd1c-7550-414a-89ce-4b223484348c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:07.756451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27fd1c-7550-414a-89ce-4b223484348c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.756763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27fd1c-7550-414a-89ce-4b223484348c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.757093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27fd1c-7550-414a-89ce-4b223484348c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.768849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27fd1c-7550-414a-89ce-4b223484348c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-5d27fd1c-7550-414a-89ce-4b223484348c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:07.777786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:07.778643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:07.779429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 252/497] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:07.780861 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a061ff2-7984-496d-8c9f-e6d59616f311 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.780861 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a061ff2-7984-496d-8c9f-e6d59616f311 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.780861 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a061ff2-7984-496d-8c9f-e6d59616f311 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.786139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 253/498] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:07.786139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a061ff2-7984-496d-8c9f-e6d59616f311 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:07.800872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:07.835797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9221431-a36a-48a0-b7c8-901b9d83afbf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:07.869412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fe393-5563-4ea5-a932-504a703442c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:07.870298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fe393-5563-4ea5-a932-504a703442c0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.870581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fe393-5563-4ea5-a932-504a703442c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.870853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fe393-5563-4ea5-a932-504a703442c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.873777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:07.877957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fe393-5563-4ea5-a932-504a703442c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:07.879405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fe393-5563-4ea5-a932-504a703442c0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:07.890501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a061ff2-7984-496d-8c9f-e6d59616f311 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-838fe393-5563-4ea5-a932-504a703442c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:07.893561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:07.894658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 246/499] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jul 21 17:21:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:07.900222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fe393-5563-4ea5-a932-504a703442c0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:07.930403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38386421-bbc7-4acc-925f-c32d6042f9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:07.931056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38386421-bbc7-4acc-925f-c32d6042f9fb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:07.931343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38386421-bbc7-4acc-925f-c32d6042f9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:07.931635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38386421-bbc7-4acc-925f-c32d6042f9fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:07.995255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fe393-5563-4ea5-a932-504a703442c0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:07.999844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38386421-bbc7-4acc-925f-c32d6042f9fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:08.001386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38386421-bbc7-4acc-925f-c32d6042f9fb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:08.022398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:08.029419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1195d-06dd-46c1-8c7f-558fcf1eb3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:08.030086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1195d-06dd-46c1-8c7f-558fcf1eb3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.030372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1195d-06dd-46c1-8c7f-558fcf1eb3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.030671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1195d-06dd-46c1-8c7f-558fcf1eb3a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.031417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1195d-06dd-46c1-8c7f-558fcf1eb3a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:08.031727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1195d-06dd-46c1-8c7f-558fcf1eb3a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:08.033124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1195d-06dd-46c1-8c7f-558fcf1eb3a4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:08.042874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 247/500] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:08.044138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a99429-a9f7-4de9-8c14-8a59c440815a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.044471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a99429-a9f7-4de9-8c14-8a59c440815a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.044751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a99429-a9f7-4de9-8c14-8a59c440815a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.061718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f1a99429-a9f7-4de9-8c14-8a59c440815a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:08.062911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f1a99429-a9f7-4de9-8c14-8a59c440815a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:08.083571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:08.100048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 254/501] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:08.101451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994740f-21b1-4ff7-9d44-9126c29aa456 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.101745 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994740f-21b1-4ff7-9d44-9126c29aa456 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.102023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994740f-21b1-4ff7-9d44-9126c29aa456 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.103282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 255/502] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:08.104665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994740f-21b1-4ff7-9d44-9126c29aa456 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:08.131915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 248/503] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:08.132708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a99429-a9f7-4de9-8c14-8a59c440815a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:08.204669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994740f-21b1-4ff7-9d44-9126c29aa456 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:08.237919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794e8bf-2a22-4960-96cc-6fe68a9e91d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:08.238581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794e8bf-2a22-4960-96cc-6fe68a9e91d2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.238859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794e8bf-2a22-4960-96cc-6fe68a9e91d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.239130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794e8bf-2a22-4960-96cc-6fe68a9e91d2 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.239661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a99429-a9f7-4de9-8c14-8a59c440815a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:08.239960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794e8bf-2a22-4960-96cc-6fe68a9e91d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:08.240267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794e8bf-2a22-4960-96cc-6fe68a9e91d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:08.241743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794e8bf-2a22-4960-96cc-6fe68a9e91d2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:08.261852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:08.273117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab560275-8087-41c4-b30a-5e50c9c9a2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:08.273902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab560275-8087-41c4-b30a-5e50c9c9a2cc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.274182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab560275-8087-41c4-b30a-5e50c9c9a2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.274454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab560275-8087-41c4-b30a-5e50c9c9a2cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.278906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 256/504] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jul 21 17:21:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:08.279730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794e8bf-2a22-4960-96cc-6fe68a9e91d2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:08.287344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab560275-8087-41c4-b30a-5e50c9c9a2cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:08.288811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab560275-8087-41c4-b30a-5e50c9c9a2cc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-ab560275-8087-41c4-b30a-5e50c9c9a2cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:08.305996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:08.307034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:08.307937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:08.307937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:08.307937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:08.307937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:08.307937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 249/505] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:08.308627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe97e7-92f3-44b4-b7b6-b2c89eb7be34 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.308731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe97e7-92f3-44b4-b7b6-b2c89eb7be34 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.308994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe97e7-92f3-44b4-b7b6-b2c89eb7be34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.325562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-15fe97e7-92f3-44b4-b7b6-b2c89eb7be34 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:08.326742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-15fe97e7-92f3-44b4-b7b6-b2c89eb7be34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:08.330432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:08.388955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794e8bf-2a22-4960-96cc-6fe68a9e91d2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:08.393057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 250/506] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:08.393916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe97e7-92f3-44b4-b7b6-b2c89eb7be34 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:08.424415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac5333-7655-4f74-adac-ab0ec2cf4343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:08.425205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac5333-7655-4f74-adac-ab0ec2cf4343 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.425474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac5333-7655-4f74-adac-ab0ec2cf4343 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.425751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac5333-7655-4f74-adac-ab0ec2cf4343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.435718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac5333-7655-4f74-adac-ab0ec2cf4343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:08.437207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac5333-7655-4f74-adac-ab0ec2cf4343 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-d0ac5333-7655-4f74-adac-ab0ec2cf4343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:08.455716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:08.456690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:08.458049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:08.458049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:08.458049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:08.458049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:08.458049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 257/507] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:08.458049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac5333-7655-4f74-adac-ab0ec2cf4343 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:08.498167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe97e7-92f3-44b4-b7b6-b2c89eb7be34 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:08.500472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:08.532189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c50ec0-3815-4c4b-9e89-f0f93df73360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:08.533005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c50ec0-3815-4c4b-9e89-f0f93df73360 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.533284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c50ec0-3815-4c4b-9e89-f0f93df73360 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.533549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c50ec0-3815-4c4b-9e89-f0f93df73360 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.543583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c50ec0-3815-4c4b-9e89-f0f93df73360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:08.550138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a0c50ec0-3815-4c4b-9e89-f0f93df73360 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:08.551425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:08.552528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:08.553501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 251/508] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:08.558674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c50ec0-3815-4c4b-9e89-f0f93df73360 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:08.558957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac5333-7655-4f74-adac-ab0ec2cf4343 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:08.597427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4a484-8b0c-416f-8972-e95220af20fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:08.598079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4a484-8b0c-416f-8972-e95220af20fe None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.598367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4a484-8b0c-416f-8972-e95220af20fe None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.598646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4a484-8b0c-416f-8972-e95220af20fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.658586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c50ec0-3815-4c4b-9e89-f0f93df73360 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:08.667590 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4a484-8b0c-416f-8972-e95220af20fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:08.669401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4a484-8b0c-416f-8972-e95220af20fe None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:08.691244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304238b5-ac53-4fcc-98a3-c913ea730395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:08.691892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304238b5-ac53-4fcc-98a3-c913ea730395 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.692172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304238b5-ac53-4fcc-98a3-c913ea730395 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.692474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304238b5-ac53-4fcc-98a3-c913ea730395 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.693195 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304238b5-ac53-4fcc-98a3-c913ea730395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:08.693495 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304238b5-ac53-4fcc-98a3-c913ea730395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:08.694857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304238b5-ac53-4fcc-98a3-c913ea730395 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:08.706907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 252/509] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:08.707690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304238b5-ac53-4fcc-98a3-c913ea730395 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:08.732494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:08.766211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:08.774560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 258/510] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:08.775409 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4a484-8b0c-416f-8972-e95220af20fe None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:08.811262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304238b5-ac53-4fcc-98a3-c913ea730395 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:08.845248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb22c5-de0a-48dc-a434-01802f53db22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:08.846036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb22c5-de0a-48dc-a434-01802f53db22 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.846315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb22c5-de0a-48dc-a434-01802f53db22 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.846589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb22c5-de0a-48dc-a434-01802f53db22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.856505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb22c5-de0a-48dc-a434-01802f53db22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:08.857947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb22c5-de0a-48dc-a434-01802f53db22 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:08.872113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4a484-8b0c-416f-8972-e95220af20fe None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e2bb22c5-de0a-48dc-a434-01802f53db22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:08.875496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:08.876481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:08.877350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:08.877350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:08.877350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:08.877350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:08.877350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 253/511] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:08.877615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb22c5-de0a-48dc-a434-01802f53db22 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:08.906405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76737d1-5759-401b-8a8a-3829f718bba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:08.907060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76737d1-5759-401b-8a8a-3829f718bba6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:08.907341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76737d1-5759-401b-8a8a-3829f718bba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:08.907614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76737d1-5759-401b-8a8a-3829f718bba6 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:08.908362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76737d1-5759-401b-8a8a-3829f718bba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:08.908738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76737d1-5759-401b-8a8a-3829f718bba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:08.910043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76737d1-5759-401b-8a8a-3829f718bba6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:08.921810 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 259/512] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:08.922626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76737d1-5759-401b-8a8a-3829f718bba6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:08.963191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:08.977112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:08.987753 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb22c5-de0a-48dc-a434-01802f53db22 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.021227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b47a05-3184-437f-adfc-56fb65ce18d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.021893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b47a05-3184-437f-adfc-56fb65ce18d0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.022175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b47a05-3184-437f-adfc-56fb65ce18d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.022449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b47a05-3184-437f-adfc-56fb65ce18d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.023094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76737d1-5759-401b-8a8a-3829f718bba6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.023439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b47a05-3184-437f-adfc-56fb65ce18d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:09.023521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b47a05-3184-437f-adfc-56fb65ce18d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.024872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b47a05-3184-437f-adfc-56fb65ce18d0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.032984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 254/513] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:09.033786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b47a05-3184-437f-adfc-56fb65ce18d0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.061396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f840765-281b-4174-98fe-8d4efb712d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.062190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840765-281b-4174-98fe-8d4efb712d73 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.062474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840765-281b-4174-98fe-8d4efb712d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.062748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840765-281b-4174-98fe-8d4efb712d73 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.070111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f840765-281b-4174-98fe-8d4efb712d73 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.071584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f840765-281b-4174-98fe-8d4efb712d73 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-5f840765-281b-4174-98fe-8d4efb712d73 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:09.087333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:09.088601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 260/514] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:09.089663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f840765-281b-4174-98fe-8d4efb712d73 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.131727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b47a05-3184-437f-adfc-56fb65ce18d0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.164062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ad7cd-d483-40a8-9e5b-0f64142eb96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.164807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ad7cd-d483-40a8-9e5b-0f64142eb96c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.165089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ad7cd-d483-40a8-9e5b-0f64142eb96c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.165362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ad7cd-d483-40a8-9e5b-0f64142eb96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.172412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:09.177563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ad7cd-d483-40a8-9e5b-0f64142eb96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.178997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ad7cd-d483-40a8-9e5b-0f64142eb96c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.191937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f840765-281b-4174-98fe-8d4efb712d73 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-d09ad7cd-d483-40a8-9e5b-0f64142eb96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:09.196458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:09.197308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:09.198151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:09.198151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:09.198151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:09.198151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:09.198151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 255/515] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:09.198695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ad7cd-d483-40a8-9e5b-0f64142eb96c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.209725 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:09.231475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cece4f-33e2-42f1-95c1-66a78bc22490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.232143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cece4f-33e2-42f1-95c1-66a78bc22490 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.232461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cece4f-33e2-42f1-95c1-66a78bc22490 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.232742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cece4f-33e2-42f1-95c1-66a78bc22490 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.233461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cece4f-33e2-42f1-95c1-66a78bc22490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:09.233771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cece4f-33e2-42f1-95c1-66a78bc22490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.235145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cece4f-33e2-42f1-95c1-66a78bc22490 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.247228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 261/516] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:09.248145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cece4f-33e2-42f1-95c1-66a78bc22490 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.299337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ad7cd-d483-40a8-9e5b-0f64142eb96c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.332774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf4583-f524-44a6-a35d-e3d3cd5d7040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.333564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf4583-f524-44a6-a35d-e3d3cd5d7040 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.333840 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf4583-f524-44a6-a35d-e3d3cd5d7040 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.334111 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf4583-f524-44a6-a35d-e3d3cd5d7040 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.341857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf4583-f524-44a6-a35d-e3d3cd5d7040 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.343323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf4583-f524-44a6-a35d-e3d3cd5d7040 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.348138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cece4f-33e2-42f1-95c1-66a78bc22490 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6aaf4583-f524-44a6-a35d-e3d3cd5d7040 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:09.358531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:09.359373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:09.360205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:09.360205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:09.360205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:09.360205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:09.360205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:09.360205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:09.360205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:09.360205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:09.360205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 256/517] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:09.360714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf4583-f524-44a6-a35d-e3d3cd5d7040 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.390499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93586cd-2bca-4d70-a61a-899e8fbcdea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.391314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93586cd-2bca-4d70-a61a-899e8fbcdea8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.391597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93586cd-2bca-4d70-a61a-899e8fbcdea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.391872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93586cd-2bca-4d70-a61a-899e8fbcdea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.402918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93586cd-2bca-4d70-a61a-899e8fbcdea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.404387 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93586cd-2bca-4d70-a61a-899e8fbcdea8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.409743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-d93586cd-2bca-4d70-a61a-899e8fbcdea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:09.422808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:09.423666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:09.424577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:09.424577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:09.424577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:09.424577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:09.424577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 262/518] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:09.424843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93586cd-2bca-4d70-a61a-899e8fbcdea8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.451518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:09.465492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf4583-f524-44a6-a35d-e3d3cd5d7040 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.500360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525a7d6-6fba-4467-b8dd-e6e903a89aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.501036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525a7d6-6fba-4467-b8dd-e6e903a89aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.501321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525a7d6-6fba-4467-b8dd-e6e903a89aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.501598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525a7d6-6fba-4467-b8dd-e6e903a89aa2 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.502442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525a7d6-6fba-4467-b8dd-e6e903a89aa2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:09.502753 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525a7d6-6fba-4467-b8dd-e6e903a89aa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.504204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525a7d6-6fba-4467-b8dd-e6e903a89aa2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.521261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 257/519] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jul 21 17:21:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:09.521884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525a7d6-6fba-4467-b8dd-e6e903a89aa2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.528076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93586cd-2bca-4d70-a61a-899e8fbcdea8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.562541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209f793-7442-40be-9338-a31694c45f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.563225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209f793-7442-40be-9338-a31694c45f61 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.563507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209f793-7442-40be-9338-a31694c45f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.563782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209f793-7442-40be-9338-a31694c45f61 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.564530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209f793-7442-40be-9338-a31694c45f61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:09.564838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209f793-7442-40be-9338-a31694c45f61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.566228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209f793-7442-40be-9338-a31694c45f61 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.577933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 263/520] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:09.578727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209f793-7442-40be-9338-a31694c45f61 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.626044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525a7d6-6fba-4467-b8dd-e6e903a89aa2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.635649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:09.661031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6df25d-c451-41fb-a5be-021d1c0aab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.661696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6df25d-c451-41fb-a5be-021d1c0aab01 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.661975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6df25d-c451-41fb-a5be-021d1c0aab01 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.662248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6df25d-c451-41fb-a5be-021d1c0aab01 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.662385 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:09.662937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6df25d-c451-41fb-a5be-021d1c0aab01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.664688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6df25d-c451-41fb-a5be-021d1c0aab01 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.678880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209f793-7442-40be-9338-a31694c45f61 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.684053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 258/521] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:21:09 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:09.684898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6df25d-c451-41fb-a5be-021d1c0aab01 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.718688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e8e37-4624-4380-af9d-98e4ef1dee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.719445 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e8e37-4624-4380-af9d-98e4ef1dee7c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.719725 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e8e37-4624-4380-af9d-98e4ef1dee7c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.720001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e8e37-4624-4380-af9d-98e4ef1dee7c None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.720845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e8e37-4624-4380-af9d-98e4ef1dee7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:09.721156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e8e37-4624-4380-af9d-98e4ef1dee7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.722925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e8e37-4624-4380-af9d-98e4ef1dee7c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.749354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 264/522] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:09.749354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e8e37-4624-4380-af9d-98e4ef1dee7c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.803613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6df25d-c451-41fb-a5be-021d1c0aab01 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.850009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1b19d-03ae-41d7-beef-624e6d37c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.851189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1b19d-03ae-41d7-beef-624e6d37c231 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.851189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1b19d-03ae-41d7-beef-624e6d37c231 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.851189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1b19d-03ae-41d7-beef-624e6d37c231 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.851884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1b19d-03ae-41d7-beef-624e6d37c231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:09.852193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1b19d-03ae-41d7-beef-624e6d37c231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.853664 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1b19d-03ae-41d7-beef-624e6d37c231 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:09.866283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 259/523] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:09.866917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1b19d-03ae-41d7-beef-624e6d37c231 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:09.873486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e8e37-4624-4380-af9d-98e4ef1dee7c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.895163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:09.917628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221488e-849f-477a-9b04-8aaa572691f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:09.918442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221488e-849f-477a-9b04-8aaa572691f7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:09.918731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221488e-849f-477a-9b04-8aaa572691f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:09.919011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221488e-849f-477a-9b04-8aaa572691f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:09.937248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:09.982989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1b19d-03ae-41d7-beef-624e6d37c231 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:09.996211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221488e-849f-477a-9b04-8aaa572691f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:09.997714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221488e-849f-477a-9b04-8aaa572691f7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:10.019943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ccbba-9da4-414b-b61c-0f18f1e74003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:10.020776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ccbba-9da4-414b-b61c-0f18f1e74003 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:10.021057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ccbba-9da4-414b-b61c-0f18f1e74003 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:10.021325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ccbba-9da4-414b-b61c-0f18f1e74003 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:10.028351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ccbba-9da4-414b-b61c-0f18f1e74003 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:10.029801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ccbba-9da4-414b-b61c-0f18f1e74003 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-826ccbba-9da4-414b-b61c-0f18f1e74003 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:10.043777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:10.044829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:10.045640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:10.045640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:10.045640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:10.045640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:10.045640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:10.045640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:10.045640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:10.045640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:10.049689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 260/524] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:10.050298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ccbba-9da4-414b-b61c-0f18f1e74003 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:10.111088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 265/525] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:10.111937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221488e-849f-477a-9b04-8aaa572691f7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:10.144705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:10.162477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:10.163676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ccbba-9da4-414b-b61c-0f18f1e74003 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:10.199814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7fd39-7c30-4017-b12f-327b1f887756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:10.200417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7fd39-7c30-4017-b12f-327b1f887756 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:10.200699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7fd39-7c30-4017-b12f-327b1f887756 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:10.200973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7fd39-7c30-4017-b12f-327b1f887756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:10.201806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7fd39-7c30-4017-b12f-327b1f887756 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=9e07c59babad479cadbf74f15ee5eea9) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:10.202225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7fd39-7c30-4017-b12f-327b1f887756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:10.203750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7fd39-7c30-4017-b12f-327b1f887756 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:10.209502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 261/526] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:10 2022] GET /identity/v3/role_assignments?user.id=9e07c59babad479cadbf74f15ee5eea9&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:10.210385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7fd39-7c30-4017-b12f-327b1f887756 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:10.229695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221488e-849f-477a-9b04-8aaa572691f7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:10.282495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057d15e-350b-4a4e-b2bf-b5242a5ab75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:10.283828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057d15e-350b-4a4e-b2bf-b5242a5ab75c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:10.284544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057d15e-350b-4a4e-b2bf-b5242a5ab75c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:10.284963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057d15e-350b-4a4e-b2bf-b5242a5ab75c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:10.297135 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057d15e-350b-4a4e-b2bf-b5242a5ab75c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:10.300256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057d15e-350b-4a4e-b2bf-b5242a5ab75c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-d057d15e-350b-4a4e-b2bf-b5242a5ab75c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:10.319782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:10.320802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:10.321685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:10.321685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:10.321685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:10.321685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:10.321685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:10.321685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:10.321685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:10.321685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:10.321685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 17:21:10.321685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:10.325008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 266/527] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:10.326413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057d15e-350b-4a4e-b2bf-b5242a5ab75c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:10.346505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7fd39-7c30-4017-b12f-327b1f887756 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:10.397902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4a4ae-e250-444c-ba10-21fe0acafae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:10.398601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4a4ae-e250-444c-ba10-21fe0acafae7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:10.398878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4a4ae-e250-444c-ba10-21fe0acafae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:10.399181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4a4ae-e250-444c-ba10-21fe0acafae7 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:10.400541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4a4ae-e250-444c-ba10-21fe0acafae7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:10.400870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4a4ae-e250-444c-ba10-21fe0acafae7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:10.402618 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4a4ae-e250-444c-ba10-21fe0acafae7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:10.430745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 262/528] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:10.430745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4a4ae-e250-444c-ba10-21fe0acafae7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:10.443792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:10.446006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057d15e-350b-4a4e-b2bf-b5242a5ab75c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:10.456492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:10.492196 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d51bc4-259e-484b-ba6b-a183d0ad8fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:10.493023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51bc4-259e-484b-ba6b-a183d0ad8fba None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:10.493302 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51bc4-259e-484b-ba6b-a183d0ad8fba None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:10.493602 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51bc4-259e-484b-ba6b-a183d0ad8fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:10.507795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d51bc4-259e-484b-ba6b-a183d0ad8fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:10.509485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d51bc4-259e-484b-ba6b-a183d0ad8fba None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-e2d51bc4-259e-484b-ba6b-a183d0ad8fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:10.532272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:10.533188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:10.534018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:10.534018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:10.534018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:10.534018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:10.536358 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 267/529] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:10.537388 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d51bc4-259e-484b-ba6b-a183d0ad8fba None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:10.550453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4a4ae-e250-444c-ba10-21fe0acafae7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:10.587916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a44c5-a68a-4f78-8945-a98ac9d4a26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:10.589099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a44c5-a68a-4f78-8945-a98ac9d4a26c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:10.589578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a44c5-a68a-4f78-8945-a98ac9d4a26c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:10.590068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a44c5-a68a-4f78-8945-a98ac9d4a26c None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:10.591206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a44c5-a68a-4f78-8945-a98ac9d4a26c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:10.591707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a44c5-a68a-4f78-8945-a98ac9d4a26c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:10.593500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a44c5-a68a-4f78-8945-a98ac9d4a26c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:10.614796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 263/530] 10.222.0.57 () {56 vars in 1139 bytes} [Thu Jul 21 17:21:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:10.615933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a44c5-a68a-4f78-8945-a98ac9d4a26c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:10.666903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d51bc4-259e-484b-ba6b-a183d0ad8fba None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:10.689811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:10.708696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d86fa-4d3d-4fbf-8515-6d66a3fae6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:10.709710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d86fa-4d3d-4fbf-8515-6d66a3fae6fc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:10.710187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d86fa-4d3d-4fbf-8515-6d66a3fae6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:10.710757 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d86fa-4d3d-4fbf-8515-6d66a3fae6fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:10.711885 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d86fa-4d3d-4fbf-8515-6d66a3fae6fc None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=380d6fe7057d44bfb20b0ae2bb1e664f) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:10.712431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d86fa-4d3d-4fbf-8515-6d66a3fae6fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:10.714404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d86fa-4d3d-4fbf-8515-6d66a3fae6fc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:10.729603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 268/531] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:10 2022] GET /identity/v3/role_assignments?user.id=380d6fe7057d44bfb20b0ae2bb1e664f&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:10.730963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d86fa-4d3d-4fbf-8515-6d66a3fae6fc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:10.731651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a44c5-a68a-4f78-8945-a98ac9d4a26c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:10.772430 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19148785-88af-48a1-baee-65a695191116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:10.773008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:10.773116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19148785-88af-48a1-baee-65a695191116 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:10.773391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19148785-88af-48a1-baee-65a695191116 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:10.773663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19148785-88af-48a1-baee-65a695191116 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:10.774398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19148785-88af-48a1-baee-65a695191116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:10.774850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19148785-88af-48a1-baee-65a695191116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:10.779408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19148785-88af-48a1-baee-65a695191116 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:10.792771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 264/532] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:10.798359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19148785-88af-48a1-baee-65a695191116 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:10.930268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19148785-88af-48a1-baee-65a695191116 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:10.965362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:10.976732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e4433-763f-4bc0-a907-7b340d1f7f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:10.977700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e4433-763f-4bc0-a907-7b340d1f7f1b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:10.977989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e4433-763f-4bc0-a907-7b340d1f7f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:10.978264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e4433-763f-4bc0-a907-7b340d1f7f1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:10.982754 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d86fa-4d3d-4fbf-8515-6d66a3fae6fc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:10.988092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e4433-763f-4bc0-a907-7b340d1f7f1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:10.989595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e4433-763f-4bc0-a907-7b340d1f7f1b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-2c6e4433-763f-4bc0-a907-7b340d1f7f1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:11.009865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:11.010758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:11.011566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:11.011566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:11.011566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:11.011566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:11.011566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:11.011566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:11.011566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:11.011566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:11.011566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 265/533] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:11.017596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e4433-763f-4bc0-a907-7b340d1f7f1b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:11.030013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3bfef-1d0e-463f-afcc-e8c49a675147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:11.030992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3bfef-1d0e-463f-afcc-e8c49a675147 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:11.031475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3bfef-1d0e-463f-afcc-e8c49a675147 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:11.031941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3bfef-1d0e-463f-afcc-e8c49a675147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:11.142070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3bfef-1d0e-463f-afcc-e8c49a675147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:11.143640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3bfef-1d0e-463f-afcc-e8c49a675147 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:11.186059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:11.194351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e4433-763f-4bc0-a907-7b340d1f7f1b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:11.276657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 269/534] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:11.282175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3bfef-1d0e-463f-afcc-e8c49a675147 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:11.293012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de1aff-edde-43e2-8286-5a50da2bdc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:11.293696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de1aff-edde-43e2-8286-5a50da2bdc8c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:11.293981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de1aff-edde-43e2-8286-5a50da2bdc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:11.294256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de1aff-edde-43e2-8286-5a50da2bdc8c None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:11.295066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de1aff-edde-43e2-8286-5a50da2bdc8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:11.295376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de1aff-edde-43e2-8286-5a50da2bdc8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:11.296850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de1aff-edde-43e2-8286-5a50da2bdc8c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:11.327726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 266/535] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:11.337324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de1aff-edde-43e2-8286-5a50da2bdc8c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:11.390189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3bfef-1d0e-463f-afcc-e8c49a675147 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:11.414864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:11.444919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5998d18-fa29-489a-9ae3-41e0e15eb193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:11.445718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5998d18-fa29-489a-9ae3-41e0e15eb193 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:11.445995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5998d18-fa29-489a-9ae3-41e0e15eb193 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:11.446262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5998d18-fa29-489a-9ae3-41e0e15eb193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:11.459837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:11.481222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5998d18-fa29-489a-9ae3-41e0e15eb193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:11.482712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5998d18-fa29-489a-9ae3-41e0e15eb193 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:11.520880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de1aff-edde-43e2-8286-5a50da2bdc8c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-e5998d18-fa29-489a-9ae3-41e0e15eb193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:11.524428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:11.525311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:11.526466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:11.526466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:11.526466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:11.526466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:11.526466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 270/536] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:11.537332 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5998d18-fa29-489a-9ae3-41e0e15eb193 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:11.561114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2c1b0-d7b5-4af4-bf10-1fd6ffb3c228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:11.562310 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2c1b0-d7b5-4af4-bf10-1fd6ffb3c228 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:11.562597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2c1b0-d7b5-4af4-bf10-1fd6ffb3c228 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:11.562870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2c1b0-d7b5-4af4-bf10-1fd6ffb3c228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:11.563701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2c1b0-d7b5-4af4-bf10-1fd6ffb3c228 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=9bbddf0ea88049aa9bb6a257e1f01ffd) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:11.564008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2c1b0-d7b5-4af4-bf10-1fd6ffb3c228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:11.565465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2c1b0-d7b5-4af4-bf10-1fd6ffb3c228 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:11.571749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 267/537] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:11 2022] GET /identity/v3/role_assignments?user.id=9bbddf0ea88049aa9bb6a257e1f01ffd&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:11.739210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5998d18-fa29-489a-9ae3-41e0e15eb193 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:11.775911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a04b91-b3b2-47fa-a28f-27abd0b5afaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:11.776605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a04b91-b3b2-47fa-a28f-27abd0b5afaf None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:11.776890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a04b91-b3b2-47fa-a28f-27abd0b5afaf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:11.777165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a04b91-b3b2-47fa-a28f-27abd0b5afaf None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:11.777890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a04b91-b3b2-47fa-a28f-27abd0b5afaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:11.778199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a04b91-b3b2-47fa-a28f-27abd0b5afaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:11.779598 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a04b91-b3b2-47fa-a28f-27abd0b5afaf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:11.791490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 271/538] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:11.796510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2c1b0-d7b5-4af4-bf10-1fd6ffb3c228 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:11.943117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2c1b0-d7b5-4af4-bf10-1fd6ffb3c228 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:11.961793 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:11.994240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805f4f2-ee7c-48d6-b772-858ab9000002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:11.995147 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805f4f2-ee7c-48d6-b772-858ab9000002 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:11.995435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805f4f2-ee7c-48d6-b772-858ab9000002 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:11.995713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805f4f2-ee7c-48d6-b772-858ab9000002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.007202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805f4f2-ee7c-48d6-b772-858ab9000002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:12.008807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805f4f2-ee7c-48d6-b772-858ab9000002 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:12.026906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82183e79-f6f1-4b1d-b036-a90cbbca3824 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:12.027557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82183e79-f6f1-4b1d-b036-a90cbbca3824 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:12.028049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82183e79-f6f1-4b1d-b036-a90cbbca3824 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.029906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 272/539] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-9805f4f2-ee7c-48d6-b772-858ab9000002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:12.030010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:12.030801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:12.031659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:12.031659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:12.031659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:12.031659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:12.031659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:12.031659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:12.031659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:12.031659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:12.031659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 268/540] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:12.040004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805f4f2-ee7c-48d6-b772-858ab9000002 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:12.042221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595b363-d136-4b7e-ac1f-46d4dfc439ba None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:12.042744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595b363-d136-4b7e-ac1f-46d4dfc439ba None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:12.043234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595b363-d136-4b7e-ac1f-46d4dfc439ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.074041 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b595b363-d136-4b7e-ac1f-46d4dfc439ba None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:12.075525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b595b363-d136-4b7e-ac1f-46d4dfc439ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:12.128675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:12.212427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 273/541] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:21:12.218169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805f4f2-ee7c-48d6-b772-858ab9000002 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:12.265151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595b363-d136-4b7e-ac1f-46d4dfc439ba None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:12.282603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac33b5-48d2-45a5-9cb4-47ef7f885d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:12.284073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac33b5-48d2-45a5-9cb4-47ef7f885d1a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:12.284380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac33b5-48d2-45a5-9cb4-47ef7f885d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:12.284660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac33b5-48d2-45a5-9cb4-47ef7f885d1a None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.285464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac33b5-48d2-45a5-9cb4-47ef7f885d1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:12.285773 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac33b5-48d2-45a5-9cb4-47ef7f885d1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:12.287218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac33b5-48d2-45a5-9cb4-47ef7f885d1a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:12.309584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 269/542] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:12.312597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac33b5-48d2-45a5-9cb4-47ef7f885d1a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:12.368567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:12.458386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595b363-d136-4b7e-ac1f-46d4dfc439ba None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:12.462515 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac33b5-48d2-45a5-9cb4-47ef7f885d1a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:12.500878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923767a-2720-495b-9ff3-cfe65fb2c3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:12.501877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923767a-2720-495b-9ff3-cfe65fb2c3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:12.502342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923767a-2720-495b-9ff3-cfe65fb2c3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:12.502797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923767a-2720-495b-9ff3-cfe65fb2c3a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.503808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923767a-2720-495b-9ff3-cfe65fb2c3a9 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=b0edda384caf46819169b69b858afc4e) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:12.504390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923767a-2720-495b-9ff3-cfe65fb2c3a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:12.506074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923767a-2720-495b-9ff3-cfe65fb2c3a9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:12.512885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 270/543] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:12 2022] GET /identity/v3/role_assignments?user.id=b0edda384caf46819169b69b858afc4e&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:12.529785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247d12e-c921-4639-929d-e868454c5c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:12.530969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247d12e-c921-4639-929d-e868454c5c99 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:12.531457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247d12e-c921-4639-929d-e868454c5c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:12.531913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247d12e-c921-4639-929d-e868454c5c99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.545470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247d12e-c921-4639-929d-e868454c5c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:12.551536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab52eb-4b3a-4958-a02d-6b5a6864c9a2 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:12.552086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab52eb-4b3a-4958-a02d-6b5a6864c9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:12.552605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab52eb-4b3a-4958-a02d-6b5a6864c9a2 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.554219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 271/544] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:12.559919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b88a0-e938-4fb6-a13c-50d571cc2530 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:12.560445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b88a0-e938-4fb6-a13c-50d571cc2530 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:12.560903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b88a0-e938-4fb6-a13c-50d571cc2530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-9247d12e-c921-4639-929d-e868454c5c99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:12.566549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:12.567505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:12.568327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:12.576847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 274/545] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:12.577949 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247d12e-c921-4639-929d-e868454c5c99 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:12.594652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-094b88a0-e938-4fb6-a13c-50d571cc2530 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:12.596186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-094b88a0-e938-4fb6-a13c-50d571cc2530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:12.645320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:12.737641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247d12e-c921-4639-929d-e868454c5c99 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:12.741746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 272/546] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:21:12.767293 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b88a0-e938-4fb6-a13c-50d571cc2530 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:12.774537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601b672-5849-4689-8ce1-aaf58eddbb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:12.776058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b672-5849-4689-8ce1-aaf58eddbb08 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:12.776574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b672-5849-4689-8ce1-aaf58eddbb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:12.777053 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b672-5849-4689-8ce1-aaf58eddbb08 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.778005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601b672-5849-4689-8ce1-aaf58eddbb08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:12.778459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601b672-5849-4689-8ce1-aaf58eddbb08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:12.780036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601b672-5849-4689-8ce1-aaf58eddbb08 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:12.789346 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 275/547] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:12.796379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601b672-5849-4689-8ce1-aaf58eddbb08 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:12.799730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:12.883230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b88a0-e938-4fb6-a13c-50d571cc2530 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:12.887574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:12.906756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601b672-5849-4689-8ce1-aaf58eddbb08 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:12.936894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b3f85-0fc9-4ed2-b204-c2b1012e0072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:12.937971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b3f85-0fc9-4ed2-b204-c2b1012e0072 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:12.938388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b3f85-0fc9-4ed2-b204-c2b1012e0072 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:12.938918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b3f85-0fc9-4ed2-b204-c2b1012e0072 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.947655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524a32a-d11b-4cec-8968-a4d9519ad34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:12.948597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524a32a-d11b-4cec-8968-a4d9519ad34d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:12.948841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524a32a-d11b-4cec-8968-a4d9519ad34d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:12.949421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524a32a-d11b-4cec-8968-a4d9519ad34d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:12.953115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b3f85-0fc9-4ed2-b204-c2b1012e0072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:12.960925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524a32a-d11b-4cec-8968-a4d9519ad34d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a21b3f85-0fc9-4ed2-b204-c2b1012e0072 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:12.961958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:12.963469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:12.964397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:12.964993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524a32a-d11b-4cec-8968-a4d9519ad34d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:12.967312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 273/548] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:12.973281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b3f85-0fc9-4ed2-b204-c2b1012e0072 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-9524a32a-d11b-4cec-8968-a4d9519ad34d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:12.989696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:12.990990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:12.992089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:12.992089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:12.992089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:12.992089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:12.996785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 276/549] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:13.005847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524a32a-d11b-4cec-8968-a4d9519ad34d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:13.073580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:13.094137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b3f85-0fc9-4ed2-b204-c2b1012e0072 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:13.165563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832c83a-33df-4f74-b255-77787266d4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:13.166648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832c83a-33df-4f74-b255-77787266d4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:13.167309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832c83a-33df-4f74-b255-77787266d4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:13.168898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832c83a-33df-4f74-b255-77787266d4a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:13.169848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832c83a-33df-4f74-b255-77787266d4a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:13.170551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832c83a-33df-4f74-b255-77787266d4a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:13.173549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832c83a-33df-4f74-b255-77787266d4a5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:13.187099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 274/550] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:13.191186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524a32a-d11b-4cec-8968-a4d9519ad34d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:13.193098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832c83a-33df-4f74-b255-77787266d4a5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:13.231031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4df41-d274-4048-9d8d-e757e703b76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:13.231707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4df41-d274-4048-9d8d-e757e703b76d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:13.231978 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4df41-d274-4048-9d8d-e757e703b76d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:13.232252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4df41-d274-4048-9d8d-e757e703b76d None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:13.233253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4df41-d274-4048-9d8d-e757e703b76d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:13.233569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4df41-d274-4048-9d8d-e757e703b76d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:13.235924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4df41-d274-4048-9d8d-e757e703b76d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:13.249410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:13.254333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 277/551] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:13.255269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4df41-d274-4048-9d8d-e757e703b76d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:13.358633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832c83a-33df-4f74-b255-77787266d4a5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:13.374229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4df41-d274-4048-9d8d-e757e703b76d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:13.405158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dc5d1-4517-4b7f-b820-9528b5f3c737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:13.406202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dc5d1-4517-4b7f-b820-9528b5f3c737 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:13.406486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dc5d1-4517-4b7f-b820-9528b5f3c737 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:13.406792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dc5d1-4517-4b7f-b820-9528b5f3c737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:13.411896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf2a4c-be34-467b-8315-d0edf4526dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:13.412986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf2a4c-be34-467b-8315-d0edf4526dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:13.413485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf2a4c-be34-467b-8315-d0edf4526dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:13.413949 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf2a4c-be34-467b-8315-d0edf4526dd2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:13.420074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dc5d1-4517-4b7f-b820-9528b5f3c737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:13.421664 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dc5d1-4517-4b7f-b820-9528b5f3c737 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:13.422749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf2a4c-be34-467b-8315-d0edf4526dd2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:13.424435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf2a4c-be34-467b-8315-d0edf4526dd2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:13.429260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-24cf2a4c-be34-467b-8315-d0edf4526dd2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:13.441190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:13.442243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:13.443146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:13.443146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:13.443146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:13.443146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:13.443146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:13.443146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:13.443146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:13.443146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:13.443146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:13.443146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8b6dc5d1-4517-4b7f-b820-9528b5f3c737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:13.443651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:13.444436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:13.445284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:13.445284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:13.445284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:13.445284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:13.447745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 275/552] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:13.448972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 278/553] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:13.449407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ecbcd-e100-42f8-9767-ea519521d851 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:13.449684 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ecbcd-e100-42f8-9767-ea519521d851 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:13.449959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ecbcd-e100-42f8-9767-ea519521d851 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:13.450791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf2a4c-be34-467b-8315-d0edf4526dd2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:13.456602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 276/554] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:13.458508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ecbcd-e100-42f8-9767-ea519521d851 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:13.519595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:13.566813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf2a4c-be34-467b-8315-d0edf4526dd2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:13.601746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ecbcd-e100-42f8-9767-ea519521d851 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:13.606807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300245e-dc2c-4021-93c2-251a169575cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:13.607822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300245e-dc2c-4021-93c2-251a169575cf None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:13.608327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300245e-dc2c-4021-93c2-251a169575cf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:13.608804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300245e-dc2c-4021-93c2-251a169575cf None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:13.609766 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300245e-dc2c-4021-93c2-251a169575cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:13.610315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300245e-dc2c-4021-93c2-251a169575cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:13.611911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300245e-dc2c-4021-93c2-251a169575cf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:13.624975 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 279/555] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:13.626560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b4363-7806-4f40-ae8c-85a66c328532 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:13.627042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b4363-7806-4f40-ae8c-85a66c328532 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:13.627537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b4363-7806-4f40-ae8c-85a66c328532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:13.646794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-715b4363-7806-4f40-ae8c-85a66c328532 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:13.648349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdb887-e968-4d7d-a295-bc358bbc4c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:13.648711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-715b4363-7806-4f40-ae8c-85a66c328532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:13.649025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdb887-e968-4d7d-a295-bc358bbc4c04 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:13.649315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdb887-e968-4d7d-a295-bc358bbc4c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:13.649579 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdb887-e968-4d7d-a295-bc358bbc4c04 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:13.651125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdb887-e968-4d7d-a295-bc358bbc4c04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:13.651522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdb887-e968-4d7d-a295-bc358bbc4c04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:13.653285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdb887-e968-4d7d-a295-bc358bbc4c04 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:13.680532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 277/556] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jul 21 17:21:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:13.681724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdb887-e968-4d7d-a295-bc358bbc4c04 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:13.727723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 280/557] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:13.728969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b4363-7806-4f40-ae8c-85a66c328532 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:13.770261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:13.801480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:13.819209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdb887-e968-4d7d-a295-bc358bbc4c04 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:13.856962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcc706-299c-47ca-8987-e405fafd3306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:13.858309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcc706-299c-47ca-8987-e405fafd3306 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:13.858787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcc706-299c-47ca-8987-e405fafd3306 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:13.859287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcc706-299c-47ca-8987-e405fafd3306 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:13.866865 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b4363-7806-4f40-ae8c-85a66c328532 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:13.868514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcc706-299c-47ca-8987-e405fafd3306 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:13.870339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcc706-299c-47ca-8987-e405fafd3306 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-34dcc706-299c-47ca-8987-e405fafd3306 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:13.885705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:13.887320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 278/558] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:13.888247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcc706-299c-47ca-8987-e405fafd3306 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:13.911774 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601cc45-a01a-41e7-a91d-679842ad81c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:13.912462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601cc45-a01a-41e7-a91d-679842ad81c7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:13.912743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601cc45-a01a-41e7-a91d-679842ad81c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:13.913572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601cc45-a01a-41e7-a91d-679842ad81c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:13.991129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601cc45-a01a-41e7-a91d-679842ad81c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:13.993187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601cc45-a01a-41e7-a91d-679842ad81c7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:14.003943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcc706-299c-47ca-8987-e405fafd3306 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.021815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:14.041532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd68e30-e7f1-44ef-8cce-66fe02db0881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:14.042563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd68e30-e7f1-44ef-8cce-66fe02db0881 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.042848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd68e30-e7f1-44ef-8cce-66fe02db0881 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.043120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd68e30-e7f1-44ef-8cce-66fe02db0881 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.046193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:14.054393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd68e30-e7f1-44ef-8cce-66fe02db0881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-fcd68e30-e7f1-44ef-8cce-66fe02db0881 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:14.062194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:14.063208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:14.064035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 279/559] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:14.065452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f28888-eed1-40fc-8d87-20bdcc133606 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.065452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f28888-eed1-40fc-8d87-20bdcc133606 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.065452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f28888-eed1-40fc-8d87-20bdcc133606 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.071106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 280/560] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:14.071106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f28888-eed1-40fc-8d87-20bdcc133606 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:14.107367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 281/561] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:14.112738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601cc45-a01a-41e7-a91d-679842ad81c7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:14.180447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f28888-eed1-40fc-8d87-20bdcc133606 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.217429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601cc45-a01a-41e7-a91d-679842ad81c7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.219617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3011c74-6fcd-4811-85a1-b2adb8220eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:14.220289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3011c74-6fcd-4811-85a1-b2adb8220eda None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.220598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3011c74-6fcd-4811-85a1-b2adb8220eda None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.220910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3011c74-6fcd-4811-85a1-b2adb8220eda None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.221745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3011c74-6fcd-4811-85a1-b2adb8220eda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:14.222059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3011c74-6fcd-4811-85a1-b2adb8220eda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:14.223482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3011c74-6fcd-4811-85a1-b2adb8220eda None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:14.241120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 281/562] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:14.242450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae2124-f0b8-4b47-9c83-72b586c04bc5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.242744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae2124-f0b8-4b47-9c83-72b586c04bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.243023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae2124-f0b8-4b47-9c83-72b586c04bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.258943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-17ae2124-f0b8-4b47-9c83-72b586c04bc5 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:14.260710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-17ae2124-f0b8-4b47-9c83-72b586c04bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:14.264349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98d8d5-1f01-4bf6-9561-b391c3a6044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:14.265546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98d8d5-1f01-4bf6-9561-b391c3a6044b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.266027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98d8d5-1f01-4bf6-9561-b391c3a6044b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.266514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98d8d5-1f01-4bf6-9561-b391c3a6044b None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.267486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98d8d5-1f01-4bf6-9561-b391c3a6044b None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:14.268494 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98d8d5-1f01-4bf6-9561-b391c3a6044b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:14.270496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98d8d5-1f01-4bf6-9561-b391c3a6044b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:14.281142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 282/563] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:14.282133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98d8d5-1f01-4bf6-9561-b391c3a6044b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:14.290787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:14.297995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:14.336974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 282/564] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:14.349283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae2124-f0b8-4b47-9c83-72b586c04bc5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:14.391660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98d8d5-1f01-4bf6-9561-b391c3a6044b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.427729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6a45e-f251-47bd-addc-835ff324175d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:14.428648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6a45e-f251-47bd-addc-835ff324175d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.428929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6a45e-f251-47bd-addc-835ff324175d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.429201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6a45e-f251-47bd-addc-835ff324175d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.440513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6a45e-f251-47bd-addc-835ff324175d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:14.442039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6a45e-f251-47bd-addc-835ff324175d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:14.453452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae2124-f0b8-4b47-9c83-72b586c04bc5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-dcd6a45e-f251-47bd-addc-835ff324175d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:14.460883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:14.462113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:14.464248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:14.464248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:14.464248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:14.464248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:14.464248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 283/565] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:14.464248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6a45e-f251-47bd-addc-835ff324175d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:14.495765 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b69fb-3e46-45d3-9cf5-a832f1943330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:14.496523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b69fb-3e46-45d3-9cf5-a832f1943330 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.496807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b69fb-3e46-45d3-9cf5-a832f1943330 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.497083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b69fb-3e46-45d3-9cf5-a832f1943330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.508082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b69fb-3e46-45d3-9cf5-a832f1943330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:14.509969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b69fb-3e46-45d3-9cf5-a832f1943330 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-ab4b69fb-3e46-45d3-9cf5-a832f1943330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:14.529489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:14.530609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:14.531703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:14.531703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:14.531703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:14.531703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:14.532356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 283/566] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:14.532593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:14.533333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b69fb-3e46-45d3-9cf5-a832f1943330 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:14.553775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:14.580444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6a45e-f251-47bd-addc-835ff324175d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.619401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1dfba-9291-4573-b469-b4e54fd69eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:14.620367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1dfba-9291-4573-b469-b4e54fd69eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.620788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1dfba-9291-4573-b469-b4e54fd69eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.621222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1dfba-9291-4573-b469-b4e54fd69eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.631890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1dfba-9291-4573-b469-b4e54fd69eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:14.633513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1dfba-9291-4573-b469-b4e54fd69eb8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:14.645416 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b69fb-3e46-45d3-9cf5-a832f1943330 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ffa1dfba-9291-4573-b469-b4e54fd69eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:14.651421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:14.652525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:14.653615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:14.653615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:14.653615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:14.653615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:14.653615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 284/567] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:14.654223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1dfba-9291-4573-b469-b4e54fd69eb8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:14.687401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6100c6d-f8b9-4666-a747-996dd8c3bea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:14.688048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6100c6d-f8b9-4666-a747-996dd8c3bea6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.688394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6100c6d-f8b9-4666-a747-996dd8c3bea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.688637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6100c6d-f8b9-4666-a747-996dd8c3bea6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.698762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6100c6d-f8b9-4666-a747-996dd8c3bea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-d6100c6d-f8b9-4666-a747-996dd8c3bea6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:14.706825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:14.707732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 284/568] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:14.709219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6100c6d-f8b9-4666-a747-996dd8c3bea6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:14.766558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1dfba-9291-4573-b469-b4e54fd69eb8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.781520 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:14.792638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:14.805913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745f3fc-8b9a-4c98-ba16-e288b59302bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:14.807594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745f3fc-8b9a-4c98-ba16-e288b59302bc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.808022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745f3fc-8b9a-4c98-ba16-e288b59302bc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.808628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745f3fc-8b9a-4c98-ba16-e288b59302bc None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.809803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745f3fc-8b9a-4c98-ba16-e288b59302bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:14.810504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745f3fc-8b9a-4c98-ba16-e288b59302bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:14.812526 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745f3fc-8b9a-4c98-ba16-e288b59302bc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:14.817564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6100c6d-f8b9-4666-a747-996dd8c3bea6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.829025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 285/569] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:14.830448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745f3fc-8b9a-4c98-ba16-e288b59302bc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:14.854940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ff23c-8ea8-41b7-8c9c-43987f5bc679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:14.855747 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ff23c-8ea8-41b7-8c9c-43987f5bc679 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.856132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ff23c-8ea8-41b7-8c9c-43987f5bc679 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.856557 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ff23c-8ea8-41b7-8c9c-43987f5bc679 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.857429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ff23c-8ea8-41b7-8c9c-43987f5bc679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:14.857832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ff23c-8ea8-41b7-8c9c-43987f5bc679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:14.859344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ff23c-8ea8-41b7-8c9c-43987f5bc679 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:14.876053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 285/570] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:14.876053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ff23c-8ea8-41b7-8c9c-43987f5bc679 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:14.941596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745f3fc-8b9a-4c98-ba16-e288b59302bc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.974828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ff23c-8ea8-41b7-8c9c-43987f5bc679 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:14.978039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38cf75-b052-4098-b54d-bc214a516aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:14.979224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38cf75-b052-4098-b54d-bc214a516aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:14.979499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38cf75-b052-4098-b54d-bc214a516aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:14.979765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38cf75-b052-4098-b54d-bc214a516aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:14.980509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38cf75-b052-4098-b54d-bc214a516aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:14.980813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38cf75-b052-4098-b54d-bc214a516aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:14.982196 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38cf75-b052-4098-b54d-bc214a516aa7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:14.993328 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 286/571] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:14.994755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38cf75-b052-4098-b54d-bc214a516aa7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.011007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18144e81-7cf7-4790-b1e0-ccc014e62d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.011852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18144e81-7cf7-4790-b1e0-ccc014e62d31 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.012174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18144e81-7cf7-4790-b1e0-ccc014e62d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.012485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18144e81-7cf7-4790-b1e0-ccc014e62d31 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.013235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18144e81-7cf7-4790-b1e0-ccc014e62d31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:15.013539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18144e81-7cf7-4790-b1e0-ccc014e62d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.014896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18144e81-7cf7-4790-b1e0-ccc014e62d31 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:15.017124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:15.023868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 286/572] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:15.024814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18144e81-7cf7-4790-b1e0-ccc014e62d31 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.032824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:15.099631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38cf75-b052-4098-b54d-bc214a516aa7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.124586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18144e81-7cf7-4790-b1e0-ccc014e62d31 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.133380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95b90-fb20-4b3c-b699-e99e3689affe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.134252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95b90-fb20-4b3c-b699-e99e3689affe None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.134661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95b90-fb20-4b3c-b699-e99e3689affe None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.135060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95b90-fb20-4b3c-b699-e99e3689affe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.141783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95b90-fb20-4b3c-b699-e99e3689affe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.143287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95b90-fb20-4b3c-b699-e99e3689affe None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:15.156596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f59caa-2a0e-4325-b885-108224657ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.157385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f59caa-2a0e-4325-b885-108224657ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.157668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f59caa-2a0e-4325-b885-108224657ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.157944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f59caa-2a0e-4325-b885-108224657ca0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-70a95b90-fb20-4b3c-b699-e99e3689affe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:15.158350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:15.159321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:15.160524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:15.160524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:15.160524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:15.160524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:15.160524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:15.160524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:15.160524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:15.160524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:15.160524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 287/573] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:15.161114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8642b9-aaec-43b5-bc36-a5fba97b2295 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.161114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8642b9-aaec-43b5-bc36-a5fba97b2295 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.161376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8642b9-aaec-43b5-bc36-a5fba97b2295 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.162668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 288/574] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:15.163442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8642b9-aaec-43b5-bc36-a5fba97b2295 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.170296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f59caa-2a0e-4325-b885-108224657ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.171745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f59caa-2a0e-4325-b885-108224657ca0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-53f59caa-2a0e-4325-b885-108224657ca0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:15.186174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:15.187229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:15.188338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:15.188338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:15.188338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:15.188338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:15.188338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:15.188338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:15.188338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:15.188338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:15.188338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 287/575] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:15.188338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f59caa-2a0e-4325-b885-108224657ca0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.248889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:15.262086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:15.270203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8642b9-aaec-43b5-bc36-a5fba97b2295 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.292452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f59caa-2a0e-4325-b885-108224657ca0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.303365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b94c4f-8357-45d0-a64a-b42e60a234e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.304151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94c4f-8357-45d0-a64a-b42e60a234e8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.304465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94c4f-8357-45d0-a64a-b42e60a234e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.304743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94c4f-8357-45d0-a64a-b42e60a234e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.312509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b94c4f-8357-45d0-a64a-b42e60a234e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.313977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b94c4f-8357-45d0-a64a-b42e60a234e8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:15.327030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0e64a-c3ee-4a68-a543-d12588a5937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.327811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0e64a-c3ee-4a68-a543-d12588a5937d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.328090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0e64a-c3ee-4a68-a543-d12588a5937d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.328382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0e64a-c3ee-4a68-a543-d12588a5937d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-52b94c4f-8357-45d0-a64a-b42e60a234e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:15.328906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:15.329822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 289/576] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jul 21 17:21:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:15.331381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b94c4f-8357-45d0-a64a-b42e60a234e8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.343133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0e64a-c3ee-4a68-a543-d12588a5937d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.344596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0e64a-c3ee-4a68-a543-d12588a5937d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-11a0e64a-c3ee-4a68-a543-d12588a5937d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:15.362047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:15.363082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:15.363976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:15.363976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:15.363976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:15.363976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:15.363976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 288/577] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:15.364561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d05f9-d104-4fae-8f0a-b2adf111e849 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.364703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d05f9-d104-4fae-8f0a-b2adf111e849 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.364982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d05f9-d104-4fae-8f0a-b2adf111e849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.381433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4d0d05f9-d104-4fae-8f0a-b2adf111e849 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:15.382602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4d0d05f9-d104-4fae-8f0a-b2adf111e849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:15.433418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b94c4f-8357-45d0-a64a-b42e60a234e8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.448729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 289/578] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:15.449809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d05f9-d104-4fae-8f0a-b2adf111e849 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.472174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e40771-c8b3-4731-b6b5-14ac1fcaa226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.473031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e40771-c8b3-4731-b6b5-14ac1fcaa226 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.473313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e40771-c8b3-4731-b6b5-14ac1fcaa226 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.473590 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e40771-c8b3-4731-b6b5-14ac1fcaa226 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.474375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e40771-c8b3-4731-b6b5-14ac1fcaa226 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:15.474685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e40771-c8b3-4731-b6b5-14ac1fcaa226 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.476110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e40771-c8b3-4731-b6b5-14ac1fcaa226 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:15.487666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 290/579] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:15.488545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e40771-c8b3-4731-b6b5-14ac1fcaa226 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.491466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:15.506064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:15.549884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d05f9-d104-4fae-8f0a-b2adf111e849 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.584934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d0910-885b-41d8-adfe-c9cc55677574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.585606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d0910-885b-41d8-adfe-c9cc55677574 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.585889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d0910-885b-41d8-adfe-c9cc55677574 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.586163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d0910-885b-41d8-adfe-c9cc55677574 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.586949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d0910-885b-41d8-adfe-c9cc55677574 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:15.587259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d0910-885b-41d8-adfe-c9cc55677574 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.588772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d0910-885b-41d8-adfe-c9cc55677574 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:15.590958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e40771-c8b3-4731-b6b5-14ac1fcaa226 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.600032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 290/580] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:15.600858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d0910-885b-41d8-adfe-c9cc55677574 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.624773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72018895-a25a-46ad-8a75-9b497a472ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.625442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72018895-a25a-46ad-8a75-9b497a472ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.625725 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72018895-a25a-46ad-8a75-9b497a472ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.625999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72018895-a25a-46ad-8a75-9b497a472ba3 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.626827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72018895-a25a-46ad-8a75-9b497a472ba3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:15.627137 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72018895-a25a-46ad-8a75-9b497a472ba3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.628594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72018895-a25a-46ad-8a75-9b497a472ba3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:15.645786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 291/581] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jul 21 17:21:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:15.646576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72018895-a25a-46ad-8a75-9b497a472ba3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.703543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:15.706725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d0910-885b-41d8-adfe-c9cc55677574 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.717384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:15.740132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee858d0-69f2-44c4-9a27-fe29ee855362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.740826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee858d0-69f2-44c4-9a27-fe29ee855362 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.741110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee858d0-69f2-44c4-9a27-fe29ee855362 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.741643 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee858d0-69f2-44c4-9a27-fe29ee855362 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.742103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee858d0-69f2-44c4-9a27-fe29ee855362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:15.742412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee858d0-69f2-44c4-9a27-fe29ee855362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.743797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee858d0-69f2-44c4-9a27-fe29ee855362 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:15.747114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72018895-a25a-46ad-8a75-9b497a472ba3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.755260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 291/582] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:15.756046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee858d0-69f2-44c4-9a27-fe29ee855362 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.784770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f220d8e-a2d0-4d4e-b7b1-de5d62785d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.785648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f220d8e-a2d0-4d4e-b7b1-de5d62785d2d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.785995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f220d8e-a2d0-4d4e-b7b1-de5d62785d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.786333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f220d8e-a2d0-4d4e-b7b1-de5d62785d2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.796169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f220d8e-a2d0-4d4e-b7b1-de5d62785d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-9f220d8e-a2d0-4d4e-b7b1-de5d62785d2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:15.803848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:15.804989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:15.805837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 292/583] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:15.806557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f220d8e-a2d0-4d4e-b7b1-de5d62785d2d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.854392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee858d0-69f2-44c4-9a27-fe29ee855362 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.888614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd129-e984-40ba-95c9-dcfefde42099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.889279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd129-e984-40ba-95c9-dcfefde42099 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.889560 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd129-e984-40ba-95c9-dcfefde42099 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.889836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd129-e984-40ba-95c9-dcfefde42099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.890668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abd129-e984-40ba-95c9-dcfefde42099 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=8a26412cf9f64b1bbbd8e595b55bcebc) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:15.890979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abd129-e984-40ba-95c9-dcfefde42099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.892424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abd129-e984-40ba-95c9-dcfefde42099 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:15.897808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 292/584] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:15 2022] GET /identity/v3/role_assignments?user.id=8a26412cf9f64b1bbbd8e595b55bcebc&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:15.898635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd129-e984-40ba-95c9-dcfefde42099 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:15.915092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f220d8e-a2d0-4d4e-b7b1-de5d62785d2d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:15.926806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:15.954805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:15.961168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475a056-1a12-4a3e-b615-df3429a48284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:15.962082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475a056-1a12-4a3e-b615-df3429a48284 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:15.962370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475a056-1a12-4a3e-b615-df3429a48284 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:15.963702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475a056-1a12-4a3e-b615-df3429a48284 None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/9e07c59babad479cadbf74f15ee5eea9/roles/b9347810c7484efd95599fb9c51ab2df` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:15.984774 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475a056-1a12-4a3e-b615-df3429a48284 None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=9e07c59babad479cadbf74f15ee5eea9, role_id=b9347810c7484efd95599fb9c51ab2df)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:15.986292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475a056-1a12-4a3e-b615-df3429a48284 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.002365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 293/585] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:21:15 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/9e07c59babad479cadbf74f15ee5eea9/roles/b9347810c7484efd95599fb9c51ab2df => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:21:16.003282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475a056-1a12-4a3e-b615-df3429a48284 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.027338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd129-e984-40ba-95c9-dcfefde42099 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.071067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb6815-a7d9-493b-8926-1d01dc4ef96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.072235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb6815-a7d9-493b-8926-1d01dc4ef96f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.072695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb6815-a7d9-493b-8926-1d01dc4ef96f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.073277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb6815-a7d9-493b-8926-1d01dc4ef96f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.100506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb6815-a7d9-493b-8926-1d01dc4ef96f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.102263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb6815-a7d9-493b-8926-1d01dc4ef96f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-2fdb6815-a7d9-493b-8926-1d01dc4ef96f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:16.126959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:16.128112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:16.129222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:16.129222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:16.129222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:16.129222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:16.129222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 293/586] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:16.129222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb6815-a7d9-493b-8926-1d01dc4ef96f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.142397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475a056-1a12-4a3e-b615-df3429a48284 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.183974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc7ce2-9d27-4622-9000-d2adca72e553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.184915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc7ce2-9d27-4622-9000-d2adca72e553 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.185207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc7ce2-9d27-4622-9000-d2adca72e553 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.185491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc7ce2-9d27-4622-9000-d2adca72e553 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.194832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc7ce2-9d27-4622-9000-d2adca72e553 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.196553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc7ce2-9d27-4622-9000-d2adca72e553 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-06cc7ce2-9d27-4622-9000-d2adca72e553 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:16.212866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:16.214046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:16.215185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:16.215185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:16.215185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:16.215185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:16.215185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:16.215185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:16.215185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:16.215185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:16.215185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 17:21:16.215185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:16.217363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 294/587] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:16.218338 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc7ce2-9d27-4622-9000-d2adca72e553 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.233947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:16.258782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb6815-a7d9-493b-8926-1d01dc4ef96f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.271863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:16.296380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c5b6-5d1f-4265-9fa9-48d45836b949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.297060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c5b6-5d1f-4265-9fa9-48d45836b949 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.297340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c5b6-5d1f-4265-9fa9-48d45836b949 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.297615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c5b6-5d1f-4265-9fa9-48d45836b949 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.298362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8c5b6-5d1f-4265-9fa9-48d45836b949 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:16.298663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8c5b6-5d1f-4265-9fa9-48d45836b949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.300028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8c5b6-5d1f-4265-9fa9-48d45836b949 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.309262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 294/588] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:16.310085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c5b6-5d1f-4265-9fa9-48d45836b949 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.339517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc7ce2-9d27-4622-9000-d2adca72e553 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.378502 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd1faa-bb3f-4317-a578-e73d8857427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.379449 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd1faa-bb3f-4317-a578-e73d8857427c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.379945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd1faa-bb3f-4317-a578-e73d8857427c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.380404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd1faa-bb3f-4317-a578-e73d8857427c None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.381269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd1faa-bb3f-4317-a578-e73d8857427c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:16.381717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd1faa-bb3f-4317-a578-e73d8857427c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.383248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd1faa-bb3f-4317-a578-e73d8857427c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.396835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 295/589] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:16.397900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd1faa-bb3f-4317-a578-e73d8857427c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.422610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c5b6-5d1f-4265-9fa9-48d45836b949 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.458754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f343f1-3d62-4158-a753-6e08ce7bd44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.459486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f343f1-3d62-4158-a753-6e08ce7bd44c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.459771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f343f1-3d62-4158-a753-6e08ce7bd44c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.460045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f343f1-3d62-4158-a753-6e08ce7bd44c None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.461005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f343f1-3d62-4158-a753-6e08ce7bd44c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:16.461310 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f343f1-3d62-4158-a753-6e08ce7bd44c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.463059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f343f1-3d62-4158-a753-6e08ce7bd44c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.471662 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:16.486844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 295/590] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:16.487755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f343f1-3d62-4158-a753-6e08ce7bd44c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.508170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd1faa-bb3f-4317-a578-e73d8857427c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.516616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:16.546608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8074445-0d63-488c-a92a-c37658cef81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.547890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8074445-0d63-488c-a92a-c37658cef81e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.548406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8074445-0d63-488c-a92a-c37658cef81e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.548953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8074445-0d63-488c-a92a-c37658cef81e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.562746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8074445-0d63-488c-a92a-c37658cef81e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.564659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8074445-0d63-488c-a92a-c37658cef81e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-e8074445-0d63-488c-a92a-c37658cef81e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:16.583605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:16.584792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:16.585686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:16.585686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:16.585686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:16.585686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:16.587579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 296/591] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:16.588863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8074445-0d63-488c-a92a-c37658cef81e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.593420 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f343f1-3d62-4158-a753-6e08ce7bd44c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.630010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a6663-03b8-4a34-9d38-00eee14bf44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.630811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6663-03b8-4a34-9d38-00eee14bf44a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.631092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6663-03b8-4a34-9d38-00eee14bf44a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.631367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6663-03b8-4a34-9d38-00eee14bf44a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.639297 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a6663-03b8-4a34-9d38-00eee14bf44a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.640803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a6663-03b8-4a34-9d38-00eee14bf44a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-414a6663-03b8-4a34-9d38-00eee14bf44a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:16.654901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:16.655816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:16.656725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:16.656725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:16.656725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:16.656725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:16.656725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:16.656725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:16.656725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:16.656725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:16.656725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 296/592] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:16.657201 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a6663-03b8-4a34-9d38-00eee14bf44a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.702402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8074445-0d63-488c-a92a-c37658cef81e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.716246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:16.741323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53a32a-637b-46fd-a33a-e1023a5447fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.742537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53a32a-637b-46fd-a33a-e1023a5447fe None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.743116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53a32a-637b-46fd-a33a-e1023a5447fe None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.743589 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53a32a-637b-46fd-a33a-e1023a5447fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.755425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53a32a-637b-46fd-a33a-e1023a5447fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.757363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53a32a-637b-46fd-a33a-e1023a5447fe None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.762962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a6663-03b8-4a34-9d38-00eee14bf44a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-3e53a32a-637b-46fd-a33a-e1023a5447fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:16.776704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:16.777617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:16.778517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:16.778517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:16.778517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:16.778517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:16.780654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 297/593] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:16.781732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53a32a-637b-46fd-a33a-e1023a5447fe None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.798586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b62a7-94a9-4855-a38b-145fe15a7d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.799251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b62a7-94a9-4855-a38b-145fe15a7d5a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.799531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b62a7-94a9-4855-a38b-145fe15a7d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.800107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b62a7-94a9-4855-a38b-145fe15a7d5a None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.800545 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b62a7-94a9-4855-a38b-145fe15a7d5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:16.800664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:16.800848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b62a7-94a9-4855-a38b-145fe15a7d5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.802204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b62a7-94a9-4855-a38b-145fe15a7d5a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.816149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 297/594] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:16.817045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b62a7-94a9-4855-a38b-145fe15a7d5a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.890487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53a32a-637b-46fd-a33a-e1023a5447fe None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.918271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b62a7-94a9-4855-a38b-145fe15a7d5a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:16.928587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8142b-826a-4b1d-9654-04031cc3dbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.929652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8142b-826a-4b1d-9654-04031cc3dbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.930234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8142b-826a-4b1d-9654-04031cc3dbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.930710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8142b-826a-4b1d-9654-04031cc3dbd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.931731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8142b-826a-4b1d-9654-04031cc3dbd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:16.932237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8142b-826a-4b1d-9654-04031cc3dbd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.934193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8142b-826a-4b1d-9654-04031cc3dbd9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.936431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:16.951341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 298/595] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:16.952466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8142b-826a-4b1d-9654-04031cc3dbd9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:16.956068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbf19a-898e-41fd-a5a0-1ffaba17b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:16.957165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbf19a-898e-41fd-a5a0-1ffaba17b0ec None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:16.957165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbf19a-898e-41fd-a5a0-1ffaba17b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:16.957421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbf19a-898e-41fd-a5a0-1ffaba17b0ec None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:16.958090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbf19a-898e-41fd-a5a0-1ffaba17b0ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:16.958397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbf19a-898e-41fd-a5a0-1ffaba17b0ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:16.959747 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbf19a-898e-41fd-a5a0-1ffaba17b0ec None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:16.972806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 298/596] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:16.973757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbf19a-898e-41fd-a5a0-1ffaba17b0ec None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:17.024234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:17.064085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8142b-826a-4b1d-9654-04031cc3dbd9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:17.086296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbf19a-898e-41fd-a5a0-1ffaba17b0ec None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:17.104317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415757d-16e0-4e41-8c8b-0762f7073184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:17.105468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415757d-16e0-4e41-8c8b-0762f7073184 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:17.105889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415757d-16e0-4e41-8c8b-0762f7073184 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:17.106402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415757d-16e0-4e41-8c8b-0762f7073184 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:17.114391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415757d-16e0-4e41-8c8b-0762f7073184 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:17.116209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415757d-16e0-4e41-8c8b-0762f7073184 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:17.125296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c9702-74ba-4370-bb14-c081e129a3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:17.126310 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c9702-74ba-4370-bb14-c081e129a3d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:17.126687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c9702-74ba-4370-bb14-c081e129a3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:17.127058 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c9702-74ba-4370-bb14-c081e129a3d8 None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/380d6fe7057d44bfb20b0ae2bb1e664f/roles/b9347810c7484efd95599fb9c51ab2df` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-c415757d-16e0-4e41-8c8b-0762f7073184 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:17.132898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:17.134004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:17.135898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:17.135898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:17.135898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:17.135898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:17.135898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:17.135898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:17.135898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:17.135898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:17.135898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 17:21:17.135898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:17.137844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 299/597] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:17.138786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415757d-16e0-4e41-8c8b-0762f7073184 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:17.146615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c9702-74ba-4370-bb14-c081e129a3d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=380d6fe7057d44bfb20b0ae2bb1e664f, role_id=b9347810c7484efd95599fb9c51ab2df)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:17.148326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c9702-74ba-4370-bb14-c081e129a3d8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:17.164671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 299/598] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:21:16 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/380d6fe7057d44bfb20b0ae2bb1e664f/roles/b9347810c7484efd95599fb9c51ab2df => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:21:17.166558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c9702-74ba-4370-bb14-c081e129a3d8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:17.207272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:17.274431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415757d-16e0-4e41-8c8b-0762f7073184 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:17.305052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:17.320451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8018649-1e19-45f2-817b-beef229dcc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:17.320451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8018649-1e19-45f2-817b-beef229dcc31 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:17.320451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8018649-1e19-45f2-817b-beef229dcc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:17.320451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8018649-1e19-45f2-817b-beef229dcc31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:17.326984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8018649-1e19-45f2-817b-beef229dcc31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:17.328637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8018649-1e19-45f2-817b-beef229dcc31 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:17.332147 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c9702-74ba-4370-bb14-c081e129a3d8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b8018649-1e19-45f2-817b-beef229dcc31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:17.343580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:17.344567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:17.345469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:17.345469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:17.345469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:17.345469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:17.345469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:17.345469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:17.345469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:17.345469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:17.345469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 300/599] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:17.355357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8018649-1e19-45f2-817b-beef229dcc31 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:17.397900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7c1cf-4df8-467e-bdd0-6e24ef27a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:17.398943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7c1cf-4df8-467e-bdd0-6e24ef27a55e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:17.399428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7c1cf-4df8-467e-bdd0-6e24ef27a55e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:17.399898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7c1cf-4df8-467e-bdd0-6e24ef27a55e None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:17.401864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7c1cf-4df8-467e-bdd0-6e24ef27a55e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:17.402407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7c1cf-4df8-467e-bdd0-6e24ef27a55e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:17.404063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7c1cf-4df8-467e-bdd0-6e24ef27a55e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:17.441293 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 300/600] 10.222.0.57 () {56 vars in 1139 bytes} [Thu Jul 21 17:21:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:17.446337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7c1cf-4df8-467e-bdd0-6e24ef27a55e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:17.520011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8018649-1e19-45f2-817b-beef229dcc31 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:17.577259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:17.583371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b77053-3b25-4d69-9efa-74dcc7417355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:17.584046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b77053-3b25-4d69-9efa-74dcc7417355 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:17.584342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b77053-3b25-4d69-9efa-74dcc7417355 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:17.584622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b77053-3b25-4d69-9efa-74dcc7417355 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:17.585410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b77053-3b25-4d69-9efa-74dcc7417355 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:17.585719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b77053-3b25-4d69-9efa-74dcc7417355 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:17.589884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b77053-3b25-4d69-9efa-74dcc7417355 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:17.613008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 301/601] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:17.615539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7c1cf-4df8-467e-bdd0-6e24ef27a55e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:17.624088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b77053-3b25-4d69-9efa-74dcc7417355 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:17.688820 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:17.701610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c1491-ecdb-4a5d-8d4e-21087de58943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:17.702435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c1491-ecdb-4a5d-8d4e-21087de58943 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:17.702719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c1491-ecdb-4a5d-8d4e-21087de58943 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:17.702991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c1491-ecdb-4a5d-8d4e-21087de58943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:17.729578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c1491-ecdb-4a5d-8d4e-21087de58943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:17.731039 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c1491-ecdb-4a5d-8d4e-21087de58943 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:17.734569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b77053-3b25-4d69-9efa-74dcc7417355 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-5d9c1491-ecdb-4a5d-8d4e-21087de58943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:17.764232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:17.765224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:17.766099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:17.766099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:17.766099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:17.766099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:17.770823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49c779-41c0-475c-9de0-8adbc7270be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:17.771720 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c779-41c0-475c-9de0-8adbc7270be3 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:17.772004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c779-41c0-475c-9de0-8adbc7270be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:17.772280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c779-41c0-475c-9de0-8adbc7270be3 None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/9bbddf0ea88049aa9bb6a257e1f01ffd/roles/b9347810c7484efd95599fb9c51ab2df` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:17.777630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 301/602] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:17.785289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c1491-ecdb-4a5d-8d4e-21087de58943 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:17.787883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49c779-41c0-475c-9de0-8adbc7270be3 None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=9bbddf0ea88049aa9bb6a257e1f01ffd, role_id=b9347810c7484efd95599fb9c51ab2df)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:17.789484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49c779-41c0-475c-9de0-8adbc7270be3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:17.805063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 302/603] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:21:17 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/9bbddf0ea88049aa9bb6a257e1f01ffd/roles/b9347810c7484efd95599fb9c51ab2df => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:21:17.980995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c1491-ecdb-4a5d-8d4e-21087de58943 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:18.027952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a628e-3084-4283-8dda-2d9a58b64646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:18.028654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a628e-3084-4283-8dda-2d9a58b64646 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.028939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a628e-3084-4283-8dda-2d9a58b64646 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.029217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a628e-3084-4283-8dda-2d9a58b64646 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.029954 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a628e-3084-4283-8dda-2d9a58b64646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:18.030261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a628e-3084-4283-8dda-2d9a58b64646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:18.031622 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a628e-3084-4283-8dda-2d9a58b64646 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:18.034240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:18.046285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 302/604] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:18.051715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49c779-41c0-475c-9de0-8adbc7270be3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:18.176349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:18.182571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de085310-05ef-423e-b10a-9146e2125468 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.182862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de085310-05ef-423e-b10a-9146e2125468 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.183144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de085310-05ef-423e-b10a-9146e2125468 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.183925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49c779-41c0-475c-9de0-8adbc7270be3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:18.185736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 303/605] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:18.191641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e424db5-5ab8-4abd-8669-91dce2135efe None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.192120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e424db5-5ab8-4abd-8669-91dce2135efe None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.192590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e424db5-5ab8-4abd-8669-91dce2135efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.221133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5e424db5-5ab8-4abd-8669-91dce2135efe None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:18.223263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5e424db5-5ab8-4abd-8669-91dce2135efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:18.235733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e8283-fe78-4b7b-afb4-e87a96d0dc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:18.236483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e8283-fe78-4b7b-afb4-e87a96d0dc76 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.236767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e8283-fe78-4b7b-afb4-e87a96d0dc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.237041 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e8283-fe78-4b7b-afb4-e87a96d0dc76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.246842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e8283-fe78-4b7b-afb4-e87a96d0dc76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:18.248979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e8283-fe78-4b7b-afb4-e87a96d0dc76 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-cb3e8283-fe78-4b7b-afb4-e87a96d0dc76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:18.267013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:18.268122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:18.269249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:18.269249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:18.269249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:18.269249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:18.269249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:18.269249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:18.269249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:18.269249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:18.269249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 303/606] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:18.274126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f243ce-79d2-4dd5-a670-43731f45cb0d None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.274126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f243ce-79d2-4dd5-a670-43731f45cb0d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.274126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f243ce-79d2-4dd5-a670-43731f45cb0d None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.276732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 304/607] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:18.277909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f243ce-79d2-4dd5-a670-43731f45cb0d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:18.331447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 304/608] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:21:18.340345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40ed4b-80c1-4335-8377-0f470c63d6f3 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.340345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40ed4b-80c1-4335-8377-0f470c63d6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.340345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40ed4b-80c1-4335-8377-0f470c63d6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.371701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ae40ed4b-80c1-4335-8377-0f470c63d6f3 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:18.373645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ae40ed4b-80c1-4335-8377-0f470c63d6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:18.415306 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f243ce-79d2-4dd5-a670-43731f45cb0d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:18.456320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 305/609] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:18.456320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40ed4b-80c1-4335-8377-0f470c63d6f3 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:18.477294 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fef67-fd13-4c5d-98c7-3ea24dececd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:18.478808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fef67-fd13-4c5d-98c7-3ea24dececd4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.479247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fef67-fd13-4c5d-98c7-3ea24dececd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.479672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fef67-fd13-4c5d-98c7-3ea24dececd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.480892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fef67-fd13-4c5d-98c7-3ea24dececd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:18.481632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fef67-fd13-4c5d-98c7-3ea24dececd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:18.483616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fef67-fd13-4c5d-98c7-3ea24dececd4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:18.502187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 305/610] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:18.503333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fef67-fd13-4c5d-98c7-3ea24dececd4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:18.524570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:18.533172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:18.584138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40ed4b-80c1-4335-8377-0f470c63d6f3 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:18.626239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4a595-4da1-44e8-af22-93d17d981915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:18.627152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4a595-4da1-44e8-af22-93d17d981915 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.627604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4a595-4da1-44e8-af22-93d17d981915 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.627954 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4a595-4da1-44e8-af22-93d17d981915 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.636438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fef67-fd13-4c5d-98c7-3ea24dececd4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:18.648328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4a595-4da1-44e8-af22-93d17d981915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-37d4a595-4da1-44e8-af22-93d17d981915 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:18.653831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:18.654763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:18.655621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:18.660229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 306/611] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:18.661220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4a595-4da1-44e8-af22-93d17d981915 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:18.688050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d809501-d37a-4005-8c91-56b97ff6b061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:18.689721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d809501-d37a-4005-8c91-56b97ff6b061 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.690150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d809501-d37a-4005-8c91-56b97ff6b061 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.690799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d809501-d37a-4005-8c91-56b97ff6b061 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.704157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d809501-d37a-4005-8c91-56b97ff6b061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6d809501-d37a-4005-8c91-56b97ff6b061 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:18.714626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:18.715791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:18.716859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:18.719688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 306/612] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:18.720666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d809501-d37a-4005-8c91-56b97ff6b061 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:18.775772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4a595-4da1-44e8-af22-93d17d981915 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:18.794424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:18.815840 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab174b26-acd6-4e8c-8a10-00af985d989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:18.817064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:18.818320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab174b26-acd6-4e8c-8a10-00af985d989b None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.818986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab174b26-acd6-4e8c-8a10-00af985d989b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.819525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab174b26-acd6-4e8c-8a10-00af985d989b None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/b0edda384caf46819169b69b858afc4e/roles/b9347810c7484efd95599fb9c51ab2df` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.836682 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab174b26-acd6-4e8c-8a10-00af985d989b None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=b0edda384caf46819169b69b858afc4e, role_id=b9347810c7484efd95599fb9c51ab2df)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:18.838574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab174b26-acd6-4e8c-8a10-00af985d989b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:18.843289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d809501-d37a-4005-8c91-56b97ff6b061 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:18.860655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 307/613] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:21:18 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/b0edda384caf46819169b69b858afc4e/roles/b9347810c7484efd95599fb9c51ab2df => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:21:18.862094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab174b26-acd6-4e8c-8a10-00af985d989b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:18.880773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d7fc0-d1dd-4850-b4e1-1a137bba3481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:18.882211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d7fc0-d1dd-4850-b4e1-1a137bba3481 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:18.882211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d7fc0-d1dd-4850-b4e1-1a137bba3481 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:18.882525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d7fc0-d1dd-4850-b4e1-1a137bba3481 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:18.883371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d7fc0-d1dd-4850-b4e1-1a137bba3481 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:18.883870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d7fc0-d1dd-4850-b4e1-1a137bba3481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:18.885515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d7fc0-d1dd-4850-b4e1-1a137bba3481 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:18.894620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 307/614] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:18.905563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d7fc0-d1dd-4850-b4e1-1a137bba3481 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:19.003146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab174b26-acd6-4e8c-8a10-00af985d989b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:19.043299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84061e5b-f238-4215-99b8-0cf119f89e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:19.044285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84061e5b-f238-4215-99b8-0cf119f89e08 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:19.044742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84061e5b-f238-4215-99b8-0cf119f89e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:19.045244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84061e5b-f238-4215-99b8-0cf119f89e08 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:19.046151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84061e5b-f238-4215-99b8-0cf119f89e08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:19.046606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84061e5b-f238-4215-99b8-0cf119f89e08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:19.048132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84061e5b-f238-4215-99b8-0cf119f89e08 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:19.058976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 308/615] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:19.065457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84061e5b-f238-4215-99b8-0cf119f89e08 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:19.114535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:19.143956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d7fc0-d1dd-4850-b4e1-1a137bba3481 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:19.197879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ba5f1-a8c3-48ea-8616-572c87d41d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:19.201226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ba5f1-a8c3-48ea-8616-572c87d41d74 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:19.201749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ba5f1-a8c3-48ea-8616-572c87d41d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:19.203779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ba5f1-a8c3-48ea-8616-572c87d41d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:19.218420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:19.227417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ba5f1-a8c3-48ea-8616-572c87d41d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:19.229288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ba5f1-a8c3-48ea-8616-572c87d41d74 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:19.255426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84061e5b-f238-4215-99b8-0cf119f89e08 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-8b5ba5f1-a8c3-48ea-8616-572c87d41d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:19.269405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:19.270484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:19.271420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:19.271420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:19.271420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:19.271420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:19.271420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 308/616] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:19.277047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ba5f1-a8c3-48ea-8616-572c87d41d74 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:19.297990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063ee7e-0ece-43d6-9521-799726b289a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:19.299072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063ee7e-0ece-43d6-9521-799726b289a3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:19.299499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063ee7e-0ece-43d6-9521-799726b289a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:19.299917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063ee7e-0ece-43d6-9521-799726b289a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:19.311876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063ee7e-0ece-43d6-9521-799726b289a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:19.313576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063ee7e-0ece-43d6-9521-799726b289a3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e063ee7e-0ece-43d6-9521-799726b289a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:19.331703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:19.332834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:19.333865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:19.333865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:19.333865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:19.333865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:19.335648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 309/617] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:19.345438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063ee7e-0ece-43d6-9521-799726b289a3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:19.440960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:19.463602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063ee7e-0ece-43d6-9521-799726b289a3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:19.507343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ba5f1-a8c3-48ea-8616-572c87d41d74 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:19.548547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe73d6b-d234-4e16-a7b7-3a1af31e8547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:19.549477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe73d6b-d234-4e16-a7b7-3a1af31e8547 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:19.549910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe73d6b-d234-4e16-a7b7-3a1af31e8547 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:19.550330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe73d6b-d234-4e16-a7b7-3a1af31e8547 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:19.551211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe73d6b-d234-4e16-a7b7-3a1af31e8547 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:19.551669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe73d6b-d234-4e16-a7b7-3a1af31e8547 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:19.553214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe73d6b-d234-4e16-a7b7-3a1af31e8547 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:19.574851 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 310/618] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:19.577200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ce312-022f-421b-ad34-269daa207fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:19.578047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ce312-022f-421b-ad34-269daa207fcc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:19.578478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ce312-022f-421b-ad34-269daa207fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:19.579017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ce312-022f-421b-ad34-269daa207fcc None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:19.579957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ce312-022f-421b-ad34-269daa207fcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:19.581013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe73d6b-d234-4e16-a7b7-3a1af31e8547 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:19.581493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ce312-022f-421b-ad34-269daa207fcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:19.583018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ce312-022f-421b-ad34-269daa207fcc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:19.609969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 309/619] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:19.616284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ce312-022f-421b-ad34-269daa207fcc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:19.677682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:19.715739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe73d6b-d234-4e16-a7b7-3a1af31e8547 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:19.737654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ce312-022f-421b-ad34-269daa207fcc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:19.759657 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f26ed-312b-4d9e-8906-93bbb17c74d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:19.760877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f26ed-312b-4d9e-8906-93bbb17c74d6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:19.761368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f26ed-312b-4d9e-8906-93bbb17c74d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:19.761844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f26ed-312b-4d9e-8906-93bbb17c74d6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:19.769589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f26ed-312b-4d9e-8906-93bbb17c74d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:19.771317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f26ed-312b-4d9e-8906-93bbb17c74d6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:19.774735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:19.781701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377a90b-02ef-4eb0-8b26-78f195d9adba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:19.782553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a90b-02ef-4eb0-8b26-78f195d9adba None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:19.782839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a90b-02ef-4eb0-8b26-78f195d9adba None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:19.783116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a90b-02ef-4eb0-8b26-78f195d9adba None admin] PATH_INFO: `/v3/users/swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-c17f26ed-312b-4d9e-8906-93bbb17c74d6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:19.791674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:19.792640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:19.793593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:19.793593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:19.793593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:19.793593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:19.793593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:19.793593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:19.793593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:19.793593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:19.793593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 17:21:19.793593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:19.794135 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377a90b-02ef-4eb0-8b26-78f195d9adba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:19.795786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377a90b-02ef-4eb0-8b26-78f195d9adba None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:19.802094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 311/620] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:19.803745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b2dd0-8998-435f-8141-8bf28854aa54 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:19.804244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b2dd0-8998-435f-8141-8bf28854aa54 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:19.804996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b2dd0-8998-435f-8141-8bf28854aa54 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:19.810526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 312/621] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:19.811513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b2dd0-8998-435f-8141-8bf28854aa54 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-c377a90b-02ef-4eb0-8b26-78f195d9adba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:19.812055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:19.812986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:19.814171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 310/622] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:19.816715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1ea07-7e4f-416d-a4a8-bf833a0433a3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:19.817121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1ea07-7e4f-416d-a4a8-bf833a0433a3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:19.817505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1ea07-7e4f-416d-a4a8-bf833a0433a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:19.845777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a6c1ea07-7e4f-416d-a4a8-bf833a0433a3 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:19.847465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a6c1ea07-7e4f-416d-a4a8-bf833a0433a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:19.928394 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 311/623] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:19.929625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1ea07-7e4f-416d-a4a8-bf833a0433a3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:19.957138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b2dd0-8998-435f-8141-8bf28854aa54 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:19.984993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:19.998864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf973b1-ac4b-43e1-9990-20fa667544d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:19.999747 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf973b1-ac4b-43e1-9990-20fa667544d6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.000137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf973b1-ac4b-43e1-9990-20fa667544d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.000675 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf973b1-ac4b-43e1-9990-20fa667544d6 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.001712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf973b1-ac4b-43e1-9990-20fa667544d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:20.002130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf973b1-ac4b-43e1-9990-20fa667544d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:20.003785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf973b1-ac4b-43e1-9990-20fa667544d6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:20.027559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 313/624] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jul 21 17:21:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:20.027559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf973b1-ac4b-43e1-9990-20fa667544d6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:20.054040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1ea07-7e4f-416d-a4a8-bf833a0433a3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:20.086500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:20.091139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390688e-40a6-4c3a-845e-19a6fd1c1158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:20.092038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390688e-40a6-4c3a-845e-19a6fd1c1158 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.092528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390688e-40a6-4c3a-845e-19a6fd1c1158 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.093011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390688e-40a6-4c3a-845e-19a6fd1c1158 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.094031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390688e-40a6-4c3a-845e-19a6fd1c1158 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:20.094519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390688e-40a6-4c3a-845e-19a6fd1c1158 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:20.096161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390688e-40a6-4c3a-845e-19a6fd1c1158 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:20.114981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 312/625] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jul 21 17:21:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:20.116344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390688e-40a6-4c3a-845e-19a6fd1c1158 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:20.135541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf973b1-ac4b-43e1-9990-20fa667544d6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:20.172243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38008dc1-21e4-4fcd-bace-bf8260574d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:20.173134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38008dc1-21e4-4fcd-bace-bf8260574d16 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.173422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38008dc1-21e4-4fcd-bace-bf8260574d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.173752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38008dc1-21e4-4fcd-bace-bf8260574d16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.185518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38008dc1-21e4-4fcd-bace-bf8260574d16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-38008dc1-21e4-4fcd-bace-bf8260574d16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:20.193678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:20.194695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:20.196670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:20.200551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 314/626] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:20.201666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38008dc1-21e4-4fcd-bace-bf8260574d16 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:20.228838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:20.234408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390688e-40a6-4c3a-845e-19a6fd1c1158 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:20.270577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ad59e-7bc9-43d8-a6a1-8d873e3b8090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:20.271187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ad59e-7bc9-43d8-a6a1-8d873e3b8090 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.271472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ad59e-7bc9-43d8-a6a1-8d873e3b8090 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.271750 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ad59e-7bc9-43d8-a6a1-8d873e3b8090 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.314833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38008dc1-21e4-4fcd-bace-bf8260574d16 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:20.334220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:20.346330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ad59e-7bc9-43d8-a6a1-8d873e3b8090 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:20.347983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ad59e-7bc9-43d8-a6a1-8d873e3b8090 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:20.352678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d95a4b-5114-47de-ba30-93effdc9371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:20.353674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d95a4b-5114-47de-ba30-93effdc9371a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.353953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d95a4b-5114-47de-ba30-93effdc9371a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.354224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d95a4b-5114-47de-ba30-93effdc9371a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.364836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d95a4b-5114-47de-ba30-93effdc9371a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:20.366247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d95a4b-5114-47de-ba30-93effdc9371a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-01d95a4b-5114-47de-ba30-93effdc9371a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:20.383510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:20.385512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:20.386508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:20.386508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:20.386508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:20.386508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:20.386508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 315/627] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:20.386800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d95a4b-5114-47de-ba30-93effdc9371a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:20.452688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:20.460340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 313/628] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:20.460340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770313fa-8615-4a0c-9eea-33f665a1058b None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.460340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770313fa-8615-4a0c-9eea-33f665a1058b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.460340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770313fa-8615-4a0c-9eea-33f665a1058b None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.461651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 314/629] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:20.466220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770313fa-8615-4a0c-9eea-33f665a1058b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:20.498082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d95a4b-5114-47de-ba30-93effdc9371a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:20.536015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054a4a2-846b-4cd6-9549-7a011e875571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:20.536729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054a4a2-846b-4cd6-9549-7a011e875571 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.537014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054a4a2-846b-4cd6-9549-7a011e875571 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.537292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054a4a2-846b-4cd6-9549-7a011e875571 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.538046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054a4a2-846b-4cd6-9549-7a011e875571 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:20.538367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054a4a2-846b-4cd6-9549-7a011e875571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:20.539726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054a4a2-846b-4cd6-9549-7a011e875571 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:20.548756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 316/630] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:20.553670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054a4a2-846b-4cd6-9549-7a011e875571 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:20.577557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770313fa-8615-4a0c-9eea-33f665a1058b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:20.587520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:20.614885 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3099b-2742-48e3-8c9b-035e730c1c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:20.615597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3099b-2742-48e3-8c9b-035e730c1c44 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.615884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3099b-2742-48e3-8c9b-035e730c1c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.616158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3099b-2742-48e3-8c9b-035e730c1c44 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.616909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3099b-2742-48e3-8c9b-035e730c1c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:20.617216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3099b-2742-48e3-8c9b-035e730c1c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:20.619152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3099b-2742-48e3-8c9b-035e730c1c44 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:20.636323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 315/631] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:20.636323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c6167-99c6-4b69-b517-b9fca5e0f769 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.636323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c6167-99c6-4b69-b517-b9fca5e0f769 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.636323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c6167-99c6-4b69-b517-b9fca5e0f769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.653534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-cd0c6167-99c6-4b69-b517-b9fca5e0f769 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:20.654714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cd0c6167-99c6-4b69-b517-b9fca5e0f769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:20.662569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054a4a2-846b-4cd6-9549-7a011e875571 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:20.698394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79296b39-c3c8-41bd-ae5b-f22e0c6470a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:20.699443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79296b39-c3c8-41bd-ae5b-f22e0c6470a6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.699719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79296b39-c3c8-41bd-ae5b-f22e0c6470a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.699993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79296b39-c3c8-41bd-ae5b-f22e0c6470a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.710836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79296b39-c3c8-41bd-ae5b-f22e0c6470a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:20.712177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:20.712313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79296b39-c3c8-41bd-ae5b-f22e0c6470a6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:20.732327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 316/632] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:20.732327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c6167-99c6-4b69-b517-b9fca5e0f769 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-79296b39-c3c8-41bd-ae5b-f22e0c6470a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:20.732604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:20.733746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:20.734626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:20.734626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:20.734626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:20.734626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:20.734626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 317/633] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:20.734626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79296b39-c3c8-41bd-ae5b-f22e0c6470a6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:20.829065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:20.844569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c6167-99c6-4b69-b517-b9fca5e0f769 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:20.847528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79296b39-c3c8-41bd-ae5b-f22e0c6470a6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:20.879067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f86bb6-0e9c-47a4-9fbb-7e9433d677be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:20.879937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f86bb6-0e9c-47a4-9fbb-7e9433d677be None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.880283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f86bb6-0e9c-47a4-9fbb-7e9433d677be None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.880643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f86bb6-0e9c-47a4-9fbb-7e9433d677be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.883765 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bd2a4-e556-4feb-b313-0e9c56d5482e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:20.885476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bd2a4-e556-4feb-b313-0e9c56d5482e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:20.885668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bd2a4-e556-4feb-b313-0e9c56d5482e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:20.885926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bd2a4-e556-4feb-b313-0e9c56d5482e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:20.891713 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f86bb6-0e9c-47a4-9fbb-7e9433d677be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:20.893216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bd2a4-e556-4feb-b313-0e9c56d5482e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:20.893347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f86bb6-0e9c-47a4-9fbb-7e9433d677be None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:20.895023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bd2a4-e556-4feb-b313-0e9c56d5482e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-180bd2a4-e556-4feb-b313-0e9c56d5482e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:20.909902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:20.911003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:20.917710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:20.917710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:20.917710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:20.917710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:20.917710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:20.917710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:20.917710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:20.917710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:20.917710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 318/634] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-c4f86bb6-0e9c-47a4-9fbb-7e9433d677be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:20.918415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:20.919835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:20.921476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:20.921476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:20.921476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:20.921476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:20.921847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bd2a4-e556-4feb-b313-0e9c56d5482e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:20.923385 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 317/635] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:20.924271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f86bb6-0e9c-47a4-9fbb-7e9433d677be None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:20.968758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:21.026694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bd2a4-e556-4feb-b313-0e9c56d5482e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.033158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f86bb6-0e9c-47a4-9fbb-7e9433d677be None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.063214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45b0bc-811c-4b12-a540-7ba00b1cb3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.063881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45b0bc-811c-4b12-a540-7ba00b1cb3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.064161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45b0bc-811c-4b12-a540-7ba00b1cb3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.064466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45b0bc-811c-4b12-a540-7ba00b1cb3e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.065194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45b0bc-811c-4b12-a540-7ba00b1cb3e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:21.065499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45b0bc-811c-4b12-a540-7ba00b1cb3e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:21.066866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45b0bc-811c-4b12-a540-7ba00b1cb3e0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:21.071142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc7b18-0b7a-455d-b5ea-a9db81b4a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.072094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc7b18-0b7a-455d-b5ea-a9db81b4a85e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.072498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc7b18-0b7a-455d-b5ea-a9db81b4a85e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.072871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc7b18-0b7a-455d-b5ea-a9db81b4a85e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.084191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 319/636] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:21.084191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45b0bc-811c-4b12-a540-7ba00b1cb3e0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:21.088127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:21.091639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc7b18-0b7a-455d-b5ea-a9db81b4a85e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-27cc7b18-0b7a-455d-b5ea-a9db81b4a85e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:21.100814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:21.101960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:21.103748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:21.107471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 318/637] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:21.108369 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc7b18-0b7a-455d-b5ea-a9db81b4a85e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:21.191410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45b0bc-811c-4b12-a540-7ba00b1cb3e0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.203473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:21.206857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc7b18-0b7a-455d-b5ea-a9db81b4a85e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.224510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de977575-8a3d-4532-859b-6a8af1f19d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.225166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de977575-8a3d-4532-859b-6a8af1f19d86 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.225450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de977575-8a3d-4532-859b-6a8af1f19d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.225727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de977575-8a3d-4532-859b-6a8af1f19d86 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.226522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de977575-8a3d-4532-859b-6a8af1f19d86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:21.226836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de977575-8a3d-4532-859b-6a8af1f19d86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:21.228928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de977575-8a3d-4532-859b-6a8af1f19d86 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:21.241068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 320/638] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:21.241219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f63e1-a963-475b-abc4-1e749376d73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.241762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f63e1-a963-475b-abc4-1e749376d73a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.241934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de977575-8a3d-4532-859b-6a8af1f19d86 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:21.242030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f63e1-a963-475b-abc4-1e749376d73a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.242345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f63e1-a963-475b-abc4-1e749376d73a None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.243026 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f63e1-a963-475b-abc4-1e749376d73a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:21.243336 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f63e1-a963-475b-abc4-1e749376d73a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:21.245849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f63e1-a963-475b-abc4-1e749376d73a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:21.257757 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 319/639] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:21.258549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f63e1-a963-475b-abc4-1e749376d73a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:21.303259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:21.352200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de977575-8a3d-4532-859b-6a8af1f19d86 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.365017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f63e1-a963-475b-abc4-1e749376d73a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.387484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db438303-bf2a-48b6-a2f5-83358d42c1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.388283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db438303-bf2a-48b6-a2f5-83358d42c1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.388599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db438303-bf2a-48b6-a2f5-83358d42c1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.388876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db438303-bf2a-48b6-a2f5-83358d42c1e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.395429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db438303-bf2a-48b6-a2f5-83358d42c1e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:21.396928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db438303-bf2a-48b6-a2f5-83358d42c1e8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:21.400213 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3861df4-5d7f-4d98-bd19-04eade73ef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.401323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3861df4-5d7f-4d98-bd19-04eade73ef31 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.401804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3861df4-5d7f-4d98-bd19-04eade73ef31 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.402265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3861df4-5d7f-4d98-bd19-04eade73ef31 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.403025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3861df4-5d7f-4d98-bd19-04eade73ef31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:21.403340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3861df4-5d7f-4d98-bd19-04eade73ef31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:21.404768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3861df4-5d7f-4d98-bd19-04eade73ef31 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-db438303-bf2a-48b6-a2f5-83358d42c1e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:21.410636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:21.411737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:21.412776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:21.412776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:21.412776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:21.412776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:21.412776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:21.412776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:21.412776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:21.412776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:21.412776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 321/640] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:21.413470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cde66b-5fef-4db5-a459-bff098669c17 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.413764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cde66b-5fef-4db5-a459-bff098669c17 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.414918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 320/641] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:21.415023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cde66b-5fef-4db5-a459-bff098669c17 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.415095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3861df4-5d7f-4d98-bd19-04eade73ef31 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:21.415471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 322/642] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:21.416336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cde66b-5fef-4db5-a459-bff098669c17 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:21.444472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:21.518819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cde66b-5fef-4db5-a459-bff098669c17 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.519240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3861df4-5d7f-4d98-bd19-04eade73ef31 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.541666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:21.552036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a714c0-2d39-4b8b-b622-238feed00008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.552185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8535919-8a83-4797-bcef-b2db3c05f065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.552846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a714c0-2d39-4b8b-b622-238feed00008 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.553039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535919-8a83-4797-bcef-b2db3c05f065 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.553132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a714c0-2d39-4b8b-b622-238feed00008 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.553320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535919-8a83-4797-bcef-b2db3c05f065 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.553413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a714c0-2d39-4b8b-b622-238feed00008 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.553592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535919-8a83-4797-bcef-b2db3c05f065 None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/8a26412cf9f64b1bbbd8e595b55bcebc/roles/b9347810c7484efd95599fb9c51ab2df` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.561687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a714c0-2d39-4b8b-b622-238feed00008 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:21.563117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a714c0-2d39-4b8b-b622-238feed00008 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:21.568078 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8535919-8a83-4797-bcef-b2db3c05f065 None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=8a26412cf9f64b1bbbd8e595b55bcebc, role_id=b9347810c7484efd95599fb9c51ab2df)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:21.569559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8535919-8a83-4797-bcef-b2db3c05f065 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-85a714c0-2d39-4b8b-b622-238feed00008 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:21.579037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:21.580037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 323/643] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jul 21 17:21:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:21.581144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a714c0-2d39-4b8b-b622-238feed00008 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:21.584230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 321/644] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:21:21 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/8a26412cf9f64b1bbbd8e595b55bcebc/roles/b9347810c7484efd95599fb9c51ab2df => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:21:21.585162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8535919-8a83-4797-bcef-b2db3c05f065 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:21.666659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:21.701450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8535919-8a83-4797-bcef-b2db3c05f065 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.714976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a714c0-2d39-4b8b-b622-238feed00008 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.754133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1676f9-b021-4645-8de2-bb3508adaac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.754931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1676f9-b021-4645-8de2-bb3508adaac5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.755208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1676f9-b021-4645-8de2-bb3508adaac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.755477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1676f9-b021-4645-8de2-bb3508adaac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.756207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad085044-d232-4021-9c42-47a0bb70f274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.756961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad085044-d232-4021-9c42-47a0bb70f274 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.757240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad085044-d232-4021-9c42-47a0bb70f274 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.757512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad085044-d232-4021-9c42-47a0bb70f274 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.758300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad085044-d232-4021-9c42-47a0bb70f274 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:21.758608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad085044-d232-4021-9c42-47a0bb70f274 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:21.760480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad085044-d232-4021-9c42-47a0bb70f274 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:21.766335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1676f9-b021-4645-8de2-bb3508adaac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:21.767775 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1676f9-b021-4645-8de2-bb3508adaac5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:21.772912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 324/645] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:21.774453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118399de-ec81-4561-82f2-732c9a34ca86 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.775020 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118399de-ec81-4561-82f2-732c9a34ca86 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.775461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118399de-ec81-4561-82f2-732c9a34ca86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2c1676f9-b021-4645-8de2-bb3508adaac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:21.787360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:21.788482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:21.789500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:21.789500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:21.789500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:21.789500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:21.789500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 322/646] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:21.790001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1676f9-b021-4645-8de2-bb3508adaac5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:21.795490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-118399de-ec81-4561-82f2-732c9a34ca86 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:21.796908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-118399de-ec81-4561-82f2-732c9a34ca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:21.839907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:21.869566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 325/647] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:21.870410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118399de-ec81-4561-82f2-732c9a34ca86 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:21.908443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1676f9-b021-4645-8de2-bb3508adaac5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:21.936433 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:21.956114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19665497-c664-498f-965e-977e3013acdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:21.958094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19665497-c664-498f-965e-977e3013acdb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:21.958094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19665497-c664-498f-965e-977e3013acdb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:21.958616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19665497-c664-498f-965e-977e3013acdb None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:21.960278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19665497-c664-498f-965e-977e3013acdb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:21.960931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19665497-c664-498f-965e-977e3013acdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:21.964310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19665497-c664-498f-965e-977e3013acdb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:21.988547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118399de-ec81-4561-82f2-732c9a34ca86 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:22.001153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 323/648] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jul 21 17:21:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:22.001153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19665497-c664-498f-965e-977e3013acdb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:22.025549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceafb4d-7b2b-4634-a05b-335c457d9726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:22.026198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceafb4d-7b2b-4634-a05b-335c457d9726 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:22.026477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceafb4d-7b2b-4634-a05b-335c457d9726 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:22.026738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceafb4d-7b2b-4634-a05b-335c457d9726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:22.027562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceafb4d-7b2b-4634-a05b-335c457d9726 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=9e07c59babad479cadbf74f15ee5eea9) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:22.027873 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceafb4d-7b2b-4634-a05b-335c457d9726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:22.029322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceafb4d-7b2b-4634-a05b-335c457d9726 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:22.035131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 326/649] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:21 2022] GET /identity/v3/role_assignments?user.id=9e07c59babad479cadbf74f15ee5eea9&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:22.035912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceafb4d-7b2b-4634-a05b-335c457d9726 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:22.101075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:22.129496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19665497-c664-498f-965e-977e3013acdb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:22.180525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceafb4d-7b2b-4634-a05b-335c457d9726 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:22.185651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240b3c4-3189-4ad6-a878-2ecf316eea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:22.186586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240b3c4-3189-4ad6-a878-2ecf316eea20 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:22.187021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240b3c4-3189-4ad6-a878-2ecf316eea20 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:22.187444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240b3c4-3189-4ad6-a878-2ecf316eea20 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:22.188356 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240b3c4-3189-4ad6-a878-2ecf316eea20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:22.188814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240b3c4-3189-4ad6-a878-2ecf316eea20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:22.191054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240b3c4-3189-4ad6-a878-2ecf316eea20 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:22.208323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 324/650] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:22.209537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240b3c4-3189-4ad6-a878-2ecf316eea20 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:22.226493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789e805-6feb-407d-981b-6821a963c782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:22.227312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789e805-6feb-407d-981b-6821a963c782 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:22.227741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789e805-6feb-407d-981b-6821a963c782 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:22.228723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789e805-6feb-407d-981b-6821a963c782 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:22.241690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789e805-6feb-407d-981b-6821a963c782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6789e805-6feb-407d-981b-6821a963c782 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:22.250206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:22.251353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:22.252432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 327/651] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:22.253317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789e805-6feb-407d-981b-6821a963c782 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:22.265379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:22.340603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240b3c4-3189-4ad6-a878-2ecf316eea20 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:22.377707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:22.383972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789e805-6feb-407d-981b-6821a963c782 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:22.395869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de14fe-25f6-4bc9-883f-b0c5c3eb968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:22.397596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de14fe-25f6-4bc9-883f-b0c5c3eb968f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:22.398319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de14fe-25f6-4bc9-883f-b0c5c3eb968f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:22.398835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de14fe-25f6-4bc9-883f-b0c5c3eb968f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:22.422698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce57086-4468-4b4a-a5bd-d94356b289fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:22.423488 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce57086-4468-4b4a-a5bd-d94356b289fd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:22.423767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce57086-4468-4b4a-a5bd-d94356b289fd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:22.424040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce57086-4468-4b4a-a5bd-d94356b289fd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:22.432901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce57086-4468-4b4a-a5bd-d94356b289fd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:22.434385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce57086-4468-4b4a-a5bd-d94356b289fd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6ce57086-4468-4b4a-a5bd-d94356b289fd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:22.450202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:22.451131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:22.452063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:22.452063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:22.452063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:22.452063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:22.452063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:22.452063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:22.452063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:22.452063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:22.452063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 17:21:22.452063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:22.456055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 328/652] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:22.457132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce57086-4468-4b4a-a5bd-d94356b289fd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:22.497835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de14fe-25f6-4bc9-883f-b0c5c3eb968f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:22.499384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de14fe-25f6-4bc9-883f-b0c5c3eb968f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:22.546313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:22.591965 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce57086-4468-4b4a-a5bd-d94356b289fd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:22.626434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 325/653] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:22.626434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de14fe-25f6-4bc9-883f-b0c5c3eb968f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:22.636309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf0f16-28c8-4c25-bd1b-2d91e63fb2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:22.637007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf0f16-28c8-4c25-bd1b-2d91e63fb2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:22.637289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf0f16-28c8-4c25-bd1b-2d91e63fb2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:22.637565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf0f16-28c8-4c25-bd1b-2d91e63fb2a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:22.638315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf0f16-28c8-4c25-bd1b-2d91e63fb2a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:22.638629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf0f16-28c8-4c25-bd1b-2d91e63fb2a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:22.640191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf0f16-28c8-4c25-bd1b-2d91e63fb2a8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:22.650962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 329/654] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:22.651854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf0f16-28c8-4c25-bd1b-2d91e63fb2a8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:22.680442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:22.749534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de14fe-25f6-4bc9-883f-b0c5c3eb968f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:22.787925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf0f16-28c8-4c25-bd1b-2d91e63fb2a8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:22.798201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1f945-122b-4927-a783-7cda738b4ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:22.799653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1f945-122b-4927-a783-7cda738b4ebb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:22.800572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1f945-122b-4927-a783-7cda738b4ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:22.801396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1f945-122b-4927-a783-7cda738b4ebb None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:22.802487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1f945-122b-4927-a783-7cda738b4ebb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:22.803237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1f945-122b-4927-a783-7cda738b4ebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:22.805654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1f945-122b-4927-a783-7cda738b4ebb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:22.826652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:22.828172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84b715-553a-4020-ade5-f2f40064bdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:22.829447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84b715-553a-4020-ade5-f2f40064bdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:22.830614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84b715-553a-4020-ade5-f2f40064bdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:22.830614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84b715-553a-4020-ade5-f2f40064bdf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:22.835548 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 326/655] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Jul 21 17:21:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:22.835548 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1f945-122b-4927-a783-7cda738b4ebb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:22.849845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84b715-553a-4020-ade5-f2f40064bdf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:22.851807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84b715-553a-4020-ade5-f2f40064bdf5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-9d84b715-553a-4020-ade5-f2f40064bdf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:22.871446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:22.872475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:22.873419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:22.873419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:22.873419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:22.873419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:22.875700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 330/656] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:22.876739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84b715-553a-4020-ade5-f2f40064bdf5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:22.966978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:22.967904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1f945-122b-4927-a783-7cda738b4ebb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.005936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7576e4f-c3d2-4bfc-bc56-cafe0ade7973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:23.007095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7576e4f-c3d2-4bfc-bc56-cafe0ade7973 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.007584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7576e4f-c3d2-4bfc-bc56-cafe0ade7973 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.007767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84b715-553a-4020-ade5-f2f40064bdf5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.008229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7576e4f-c3d2-4bfc-bc56-cafe0ade7973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.020030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7576e4f-c3d2-4bfc-bc56-cafe0ade7973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:23.021812 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7576e4f-c3d2-4bfc-bc56-cafe0ade7973 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-a7576e4f-c3d2-4bfc-bc56-cafe0ade7973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:23.042193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:23.043079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:23.043999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:23.043999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:23.043999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:23.043999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:23.046436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 327/657] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:23.047505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7576e4f-c3d2-4bfc-bc56-cafe0ade7973 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:23.047619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475bec4-de85-4054-9cdb-831b448a4b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:23.048337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475bec4-de85-4054-9cdb-831b448a4b81 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.048633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475bec4-de85-4054-9cdb-831b448a4b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.048917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475bec4-de85-4054-9cdb-831b448a4b81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.113384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:23.145100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475bec4-de85-4054-9cdb-831b448a4b81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:23.147259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475bec4-de85-4054-9cdb-831b448a4b81 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:23.166489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7576e4f-c3d2-4bfc-bc56-cafe0ade7973 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.203088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f56ba-f142-4fbd-8f33-0f5eb1d12f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:23.204137 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f56ba-f142-4fbd-8f33-0f5eb1d12f03 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.204653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f56ba-f142-4fbd-8f33-0f5eb1d12f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.205161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f56ba-f142-4fbd-8f33-0f5eb1d12f03 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.206080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f56ba-f142-4fbd-8f33-0f5eb1d12f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:23.206657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f56ba-f142-4fbd-8f33-0f5eb1d12f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:23.208306 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f56ba-f142-4fbd-8f33-0f5eb1d12f03 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:23.220714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 328/658] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:23.221824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f56ba-f142-4fbd-8f33-0f5eb1d12f03 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:23.235562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:23.280638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 331/659] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:23.280638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475bec4-de85-4054-9cdb-831b448a4b81 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:23.362506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f56ba-f142-4fbd-8f33-0f5eb1d12f03 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.376040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:23.399005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8507e05-68d9-4233-aa3e-9e6867956289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:23.400404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8507e05-68d9-4233-aa3e-9e6867956289 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.401111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8507e05-68d9-4233-aa3e-9e6867956289 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.401111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8507e05-68d9-4233-aa3e-9e6867956289 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.401912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475bec4-de85-4054-9cdb-831b448a4b81 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.402040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8507e05-68d9-4233-aa3e-9e6867956289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:23.402385 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8507e05-68d9-4233-aa3e-9e6867956289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:23.403863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8507e05-68d9-4233-aa3e-9e6867956289 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:23.422523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 329/660] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:23.423538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8507e05-68d9-4233-aa3e-9e6867956289 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:23.440947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e91e1-dd82-4ddd-ad0c-1ff9bb9f3466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:23.442116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e91e1-dd82-4ddd-ad0c-1ff9bb9f3466 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.442565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e91e1-dd82-4ddd-ad0c-1ff9bb9f3466 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.443069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e91e1-dd82-4ddd-ad0c-1ff9bb9f3466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.452241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e91e1-dd82-4ddd-ad0c-1ff9bb9f3466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:23.454107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e91e1-dd82-4ddd-ad0c-1ff9bb9f3466 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f27e91e1-dd82-4ddd-ad0c-1ff9bb9f3466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:23.470227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:23.471382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:23.472456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:23.472456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:23.472456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:23.472456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:23.472456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:23.472456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:23.472456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:23.472456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:23.479670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 332/661] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:23.480738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e91e1-dd82-4ddd-ad0c-1ff9bb9f3466 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:23.517817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:23.535326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8507e05-68d9-4233-aa3e-9e6867956289 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.585452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1abac-7485-47b5-b0b1-caf120f630fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:23.587251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1abac-7485-47b5-b0b1-caf120f630fa None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.588497 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1abac-7485-47b5-b0b1-caf120f630fa None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.588988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1abac-7485-47b5-b0b1-caf120f630fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.593150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e91e1-dd82-4ddd-ad0c-1ff9bb9f3466 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.602849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1abac-7485-47b5-b0b1-caf120f630fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:23.605764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1abac-7485-47b5-b0b1-caf120f630fa None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:23.629591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2378d4-e844-4c73-b324-22be3a2cd32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-21d1abac-7485-47b5-b0b1-caf120f630fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:23.630389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:23.631248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:23.632238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:23.632238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:23.632238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:23.632238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:23.632482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2378d4-e844-4c73-b324-22be3a2cd32e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.632482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2378d4-e844-4c73-b324-22be3a2cd32e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.632482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2378d4-e844-4c73-b324-22be3a2cd32e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.633654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 330/662] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:23.635371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1abac-7485-47b5-b0b1-caf120f630fa None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:23.644007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2378d4-e844-4c73-b324-22be3a2cd32e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:23.646124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2378d4-e844-4c73-b324-22be3a2cd32e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:23.661699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-fc2378d4-e844-4c73-b324-22be3a2cd32e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:23.666592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:23.667558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:23.668563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:23.668563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:23.668563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:23.668563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:23.668563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:23.668563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:23.668563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:23.668563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:23.668563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 17:21:23.668563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:23.674467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 333/663] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:23.674467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2378d4-e844-4c73-b324-22be3a2cd32e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:23.753359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1abac-7485-47b5-b0b1-caf120f630fa None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.778028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:23.789551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ce440-97da-43d3-8b21-3a54727d5f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:23.789743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2378d4-e844-4c73-b324-22be3a2cd32e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.790159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ce440-97da-43d3-8b21-3a54727d5f47 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.790478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ce440-97da-43d3-8b21-3a54727d5f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.790759 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ce440-97da-43d3-8b21-3a54727d5f47 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.791543 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ce440-97da-43d3-8b21-3a54727d5f47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:23.791858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ce440-97da-43d3-8b21-3a54727d5f47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:23.793319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ce440-97da-43d3-8b21-3a54727d5f47 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:23.810687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 331/664] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:23.810687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ce440-97da-43d3-8b21-3a54727d5f47 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:23.832001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfa586-8b07-47f7-9d5c-aedf9b6737a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:23.832779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfa586-8b07-47f7-9d5c-aedf9b6737a4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.833059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfa586-8b07-47f7-9d5c-aedf9b6737a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.833332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfa586-8b07-47f7-9d5c-aedf9b6737a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.834053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfa586-8b07-47f7-9d5c-aedf9b6737a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:23.834351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfa586-8b07-47f7-9d5c-aedf9b6737a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:23.835705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfa586-8b07-47f7-9d5c-aedf9b6737a4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:23.848160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 334/665] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:23.849458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e85ed7-d346-45dc-b814-d984bc2e035c None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.849742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e85ed7-d346-45dc-b814-d984bc2e035c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.850018 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e85ed7-d346-45dc-b814-d984bc2e035c None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.856120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 335/666] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:23.856120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e85ed7-d346-45dc-b814-d984bc2e035c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:23.906886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:23.924224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ce440-97da-43d3-8b21-3a54727d5f47 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.969539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4d71e-ad56-4641-b31b-9c3063db7fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:23.970551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4d71e-ad56-4641-b31b-9c3063db7fee None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:23.971020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4d71e-ad56-4641-b31b-9c3063db7fee None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:23.971485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4d71e-ad56-4641-b31b-9c3063db7fee None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:23.972568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4d71e-ad56-4641-b31b-9c3063db7fee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:23.972978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e85ed7-d346-45dc-b814-d984bc2e035c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:23.973282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4d71e-ad56-4641-b31b-9c3063db7fee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:23.974900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4d71e-ad56-4641-b31b-9c3063db7fee None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:23.993586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 332/667] 10.222.0.57 () {56 vars in 1139 bytes} [Thu Jul 21 17:21:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:23.995400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4d71e-ad56-4641-b31b-9c3063db7fee None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:24.017969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ecd45-dc50-4a5a-9576-f196d94a2c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:24.018729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ecd45-dc50-4a5a-9576-f196d94a2c60 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.019085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ecd45-dc50-4a5a-9576-f196d94a2c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.019431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ecd45-dc50-4a5a-9576-f196d94a2c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.020561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ecd45-dc50-4a5a-9576-f196d94a2c60 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=380d6fe7057d44bfb20b0ae2bb1e664f) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:24.020951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ecd45-dc50-4a5a-9576-f196d94a2c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:24.022710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ecd45-dc50-4a5a-9576-f196d94a2c60 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:24.040934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 336/668] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:23 2022] GET /identity/v3/role_assignments?user.id=380d6fe7057d44bfb20b0ae2bb1e664f&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:24.040934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569fcfe-1c3e-4cd3-97f4-7407b34d1990 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.040934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569fcfe-1c3e-4cd3-97f4-7407b34d1990 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.040934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569fcfe-1c3e-4cd3-97f4-7407b34d1990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.062694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d569fcfe-1c3e-4cd3-97f4-7407b34d1990 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:24.064377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d569fcfe-1c3e-4cd3-97f4-7407b34d1990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:24.080448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:24.137738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4d71e-ad56-4641-b31b-9c3063db7fee None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:24.174189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 337/669] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:24.175500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569fcfe-1c3e-4cd3-97f4-7407b34d1990 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:24.192924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f364e-fbf3-442f-ac50-334413a052af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:24.193655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f364e-fbf3-442f-ac50-334413a052af None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.193946 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f364e-fbf3-442f-ac50-334413a052af None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.194225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f364e-fbf3-442f-ac50-334413a052af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.210803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f364e-fbf3-442f-ac50-334413a052af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:24.212534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f364e-fbf3-442f-ac50-334413a052af None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-e04f364e-fbf3-442f-ac50-334413a052af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:24.229994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:24.230961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:24.232256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:24.232256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:24.232256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:24.232256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:24.232256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:24.232256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:24.232256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:24.232256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:24.232256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 333/670] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:24.232256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f364e-fbf3-442f-ac50-334413a052af None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:24.239299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:24.303690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569fcfe-1c3e-4cd3-97f4-7407b34d1990 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:24.341294 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd4730-5f82-4312-815e-a5906be038d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:24.341951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd4730-5f82-4312-815e-a5906be038d2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.342237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd4730-5f82-4312-815e-a5906be038d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.342518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd4730-5f82-4312-815e-a5906be038d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.358424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:24.363640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f364e-fbf3-442f-ac50-334413a052af None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:24.405859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d91d67-c817-4823-9194-6f3ee6269c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:24.406661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d91d67-c817-4823-9194-6f3ee6269c4f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.406941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d91d67-c817-4823-9194-6f3ee6269c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.407215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d91d67-c817-4823-9194-6f3ee6269c4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.418406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d91d67-c817-4823-9194-6f3ee6269c4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:24.420815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd4730-5f82-4312-815e-a5906be038d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:24.422271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd4730-5f82-4312-815e-a5906be038d2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-37d91d67-c817-4823-9194-6f3ee6269c4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:24.426782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:24.427763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:24.429185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:24.433665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 334/671] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:24.434278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d91d67-c817-4823-9194-6f3ee6269c4f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:24.504451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:24.527296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 338/672] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:24.528823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff798364-7c8e-4115-8e7b-5abd12bdc352 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.529181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff798364-7c8e-4115-8e7b-5abd12bdc352 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.529534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff798364-7c8e-4115-8e7b-5abd12bdc352 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.531075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 339/673] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:24.535516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff798364-7c8e-4115-8e7b-5abd12bdc352 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:24.552695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d91d67-c817-4823-9194-6f3ee6269c4f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:24.585362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:24.592500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b5c97-6a8d-448a-868c-ddd7791ceb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:24.593219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b5c97-6a8d-448a-868c-ddd7791ceb2c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.593434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b5c97-6a8d-448a-868c-ddd7791ceb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.593708 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b5c97-6a8d-448a-868c-ddd7791ceb2c None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.594488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b5c97-6a8d-448a-868c-ddd7791ceb2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:24.594796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b5c97-6a8d-448a-868c-ddd7791ceb2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:24.596213 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b5c97-6a8d-448a-868c-ddd7791ceb2c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:24.613428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 335/674] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:24.613428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b5c97-6a8d-448a-868c-ddd7791ceb2c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:24.653288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff798364-7c8e-4115-8e7b-5abd12bdc352 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:24.691024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab924c-9d4c-4cb6-8529-d8913694b1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:24.692037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab924c-9d4c-4cb6-8529-d8913694b1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.692552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab924c-9d4c-4cb6-8529-d8913694b1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.693024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab924c-9d4c-4cb6-8529-d8913694b1d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.694054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab924c-9d4c-4cb6-8529-d8913694b1d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:24.694593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab924c-9d4c-4cb6-8529-d8913694b1d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:24.696262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab924c-9d4c-4cb6-8529-d8913694b1d3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:24.706496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 340/675] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:24.707887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19e53e-6221-4d66-8d01-84553f430131 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.708168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19e53e-6221-4d66-8d01-84553f430131 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.708469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19e53e-6221-4d66-8d01-84553f430131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.723297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b5c97-6a8d-448a-868c-ddd7791ceb2c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:24.726597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7e19e53e-6221-4d66-8d01-84553f430131 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:24.727791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7e19e53e-6221-4d66-8d01-84553f430131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:24.747098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:24.760097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02a6ea-b151-4b9c-830c-806740fca29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:24.760912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02a6ea-b151-4b9c-830c-806740fca29e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.761198 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02a6ea-b151-4b9c-830c-806740fca29e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.761484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02a6ea-b151-4b9c-830c-806740fca29e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.771545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02a6ea-b151-4b9c-830c-806740fca29e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:24.773023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02a6ea-b151-4b9c-830c-806740fca29e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-8a02a6ea-b151-4b9c-830c-806740fca29e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:24.791116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:24.792340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:24.793437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:24.793437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:24.793437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:24.793437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:24.796653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 336/676] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:24.797457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02a6ea-b151-4b9c-830c-806740fca29e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:24.801982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 341/677] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:24.803300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19e53e-6221-4d66-8d01-84553f430131 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:24.855610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:24.907244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02a6ea-b151-4b9c-830c-806740fca29e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:24.922295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19e53e-6221-4d66-8d01-84553f430131 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:24.947512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67466735-2f3a-48df-8a8d-04b5e7dc3778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:24.948324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67466735-2f3a-48df-8a8d-04b5e7dc3778 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.948698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67466735-2f3a-48df-8a8d-04b5e7dc3778 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.949274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67466735-2f3a-48df-8a8d-04b5e7dc3778 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.950101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67466735-2f3a-48df-8a8d-04b5e7dc3778 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=9bbddf0ea88049aa9bb6a257e1f01ffd) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:24.950419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67466735-2f3a-48df-8a8d-04b5e7dc3778 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:24.951839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67466735-2f3a-48df-8a8d-04b5e7dc3778 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:24.958956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 337/678] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:24 2022] GET /identity/v3/role_assignments?user.id=9bbddf0ea88049aa9bb6a257e1f01ffd&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:24.968567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67466735-2f3a-48df-8a8d-04b5e7dc3778 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:24.973288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c89c94-53bd-4496-8123-baee5070c1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:24.974253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c89c94-53bd-4496-8123-baee5070c1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:24.974657 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c89c94-53bd-4496-8123-baee5070c1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:24.975013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c89c94-53bd-4496-8123-baee5070c1c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:24.989239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c89c94-53bd-4496-8123-baee5070c1c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:24.990813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c89c94-53bd-4496-8123-baee5070c1c1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f9c89c94-53bd-4496-8123-baee5070c1c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:25.009966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:25.010946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:25.011828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:25.011828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:25.011828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:25.011828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:25.012053 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:25.012128 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 342/679] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:25.012128 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c89c94-53bd-4496-8123-baee5070c1c1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:25.100689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67466735-2f3a-48df-8a8d-04b5e7dc3778 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:25.125069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:25.137216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c89c94-53bd-4496-8123-baee5070c1c1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:25.149001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf21b7-9d5e-4b56-a5af-0d87b3457f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:25.150154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf21b7-9d5e-4b56-a5af-0d87b3457f18 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:25.150612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf21b7-9d5e-4b56-a5af-0d87b3457f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:25.151071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf21b7-9d5e-4b56-a5af-0d87b3457f18 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:25.151965 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf21b7-9d5e-4b56-a5af-0d87b3457f18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:25.152613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf21b7-9d5e-4b56-a5af-0d87b3457f18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:25.154287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf21b7-9d5e-4b56-a5af-0d87b3457f18 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:25.168526 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 338/680] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:25.169269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf21b7-9d5e-4b56-a5af-0d87b3457f18 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:25.181428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b55f0-44bc-4f29-b512-75a4655763b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:25.182411 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b55f0-44bc-4f29-b512-75a4655763b3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:25.182688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b55f0-44bc-4f29-b512-75a4655763b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:25.182958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b55f0-44bc-4f29-b512-75a4655763b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:25.195052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b55f0-44bc-4f29-b512-75a4655763b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:25.196914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b55f0-44bc-4f29-b512-75a4655763b3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f22b55f0-44bc-4f29-b512-75a4655763b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:25.217656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:25.218748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:25.219624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:25.219624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:25.219624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:25.219624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:25.221280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 343/681] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:25.222267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b55f0-44bc-4f29-b512-75a4655763b3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:25.291747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:25.296065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf21b7-9d5e-4b56-a5af-0d87b3457f18 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:25.332610 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce28dfd-1f20-4199-af8a-5877a6b11330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:25.333570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28dfd-1f20-4199-af8a-5877a6b11330 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:25.333990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28dfd-1f20-4199-af8a-5877a6b11330 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:25.334434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28dfd-1f20-4199-af8a-5877a6b11330 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:25.335297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce28dfd-1f20-4199-af8a-5877a6b11330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:25.335744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce28dfd-1f20-4199-af8a-5877a6b11330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:25.337864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce28dfd-1f20-4199-af8a-5877a6b11330 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:25.351040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 339/682] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:25.356326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce28dfd-1f20-4199-af8a-5877a6b11330 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:25.364417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b55f0-44bc-4f29-b512-75a4655763b3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:25.412195 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8c58-6610-4d46-89cd-c5cf6f5f700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:25.413500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8c58-6610-4d46-89cd-c5cf6f5f700f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:25.413804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8c58-6610-4d46-89cd-c5cf6f5f700f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:25.414092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8c58-6610-4d46-89cd-c5cf6f5f700f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:25.424649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d8c58-6610-4d46-89cd-c5cf6f5f700f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:25.427093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d8c58-6610-4d46-89cd-c5cf6f5f700f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-ee7d8c58-6610-4d46-89cd-c5cf6f5f700f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:25.444446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:25.445483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:25.446376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:25.446376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:25.446376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:25.446376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:25.446376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:25.446376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:25.446376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:25.446376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:25.446376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 344/683] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:25.447485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8c58-6610-4d46-89cd-c5cf6f5f700f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:25.450841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:25.470348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce28dfd-1f20-4199-af8a-5877a6b11330 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:25.505648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ad113-d155-45e6-8e03-26cc02e3cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:25.506718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ad113-d155-45e6-8e03-26cc02e3cbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:25.507168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ad113-d155-45e6-8e03-26cc02e3cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:25.507581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ad113-d155-45e6-8e03-26cc02e3cbf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:25.508474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ad113-d155-45e6-8e03-26cc02e3cbf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:25.508935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ad113-d155-45e6-8e03-26cc02e3cbf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:25.511514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ad113-d155-45e6-8e03-26cc02e3cbf0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:25.524753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 340/684] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:25.525967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ad113-d155-45e6-8e03-26cc02e3cbf0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:25.547039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:25.560803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8c58-6610-4d46-89cd-c5cf6f5f700f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:25.598289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa00dda-739e-4d44-a389-f00341f6d3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:25.599339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa00dda-739e-4d44-a389-f00341f6d3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:25.599722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa00dda-739e-4d44-a389-f00341f6d3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:25.600100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa00dda-739e-4d44-a389-f00341f6d3f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:25.608772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa00dda-739e-4d44-a389-f00341f6d3f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:25.610377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa00dda-739e-4d44-a389-f00341f6d3f0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1fa00dda-739e-4d44-a389-f00341f6d3f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:25.626377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:25.627591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:25.628790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:25.628790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:25.628790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:25.628790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:25.628790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:25.628790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:25.628790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:25.628790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:25.628790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:25.628790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:25.635862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 345/685] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:25.636868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa00dda-739e-4d44-a389-f00341f6d3f0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:25.641220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ad113-d155-45e6-8e03-26cc02e3cbf0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:25.678167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c352cb6-9974-47d4-abb0-42b967782ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:25.679048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352cb6-9974-47d4-abb0-42b967782ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:25.679488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352cb6-9974-47d4-abb0-42b967782ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:25.679912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352cb6-9974-47d4-abb0-42b967782ee4 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:25.681605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c352cb6-9974-47d4-abb0-42b967782ee4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:25.682060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c352cb6-9974-47d4-abb0-42b967782ee4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:25.683785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c352cb6-9974-47d4-abb0-42b967782ee4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:25.697225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 341/686] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:25.698359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c352cb6-9974-47d4-abb0-42b967782ee4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:25.704636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:25.748665 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa00dda-739e-4d44-a389-f00341f6d3f0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:25.785206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba3b0-cad6-413b-a468-bbc54529e39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:25.785986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba3b0-cad6-413b-a468-bbc54529e39f None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:25.786375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba3b0-cad6-413b-a468-bbc54529e39f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:25.786777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba3b0-cad6-413b-a468-bbc54529e39f None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:25.787312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:25.788954 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffba3b0-cad6-413b-a468-bbc54529e39f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:25.791480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffba3b0-cad6-413b-a468-bbc54529e39f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:25.812005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c352cb6-9974-47d4-abb0-42b967782ee4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:25.814192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 346/687] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:21:25 2022] POST /identity/v3/users => generated 254 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:25.815295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba3b0-cad6-413b-a468-bbc54529e39f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:25.854054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34f904-bad3-4794-b6d8-aad324e88f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:25.854906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34f904-bad3-4794-b6d8-aad324e88f9f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:25.855188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34f904-bad3-4794-b6d8-aad324e88f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:25.855462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34f904-bad3-4794-b6d8-aad324e88f9f None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:25.856463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34f904-bad3-4794-b6d8-aad324e88f9f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:25.856871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34f904-bad3-4794-b6d8-aad324e88f9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:25.858365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34f904-bad3-4794-b6d8-aad324e88f9f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:25.879247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 342/688] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jul 21 17:21:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:25.886520 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34f904-bad3-4794-b6d8-aad324e88f9f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:25.940335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba3b0-cad6-413b-a468-bbc54529e39f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:25.956071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:26.016007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eef98e-3fc8-4b87-b730-4c06c4103a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:26.017103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eef98e-3fc8-4b87-b730-4c06c4103a87 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:26.017583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eef98e-3fc8-4b87-b730-4c06c4103a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:26.018073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eef98e-3fc8-4b87-b730-4c06c4103a87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:26.019112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eef98e-3fc8-4b87-b730-4c06c4103a87 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=b0edda384caf46819169b69b858afc4e) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:26.019632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eef98e-3fc8-4b87-b730-4c06c4103a87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:26.021286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eef98e-3fc8-4b87-b730-4c06c4103a87 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:26.039666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34f904-bad3-4794-b6d8-aad324e88f9f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:26.040025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 347/689] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:25 2022] GET /identity/v3/role_assignments?user.id=b0edda384caf46819169b69b858afc4e&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:26.079548 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673034b3-5795-4105-8a6c-d4541eb7f361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:26.080215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673034b3-5795-4105-8a6c-d4541eb7f361 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:26.080551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673034b3-5795-4105-8a6c-d4541eb7f361 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:26.080825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673034b3-5795-4105-8a6c-d4541eb7f361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:26.093126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:26.158480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673034b3-5795-4105-8a6c-d4541eb7f361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:26.159979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673034b3-5795-4105-8a6c-d4541eb7f361 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:26.272333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 343/690] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:26.278908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eef98e-3fc8-4b87-b730-4c06c4103a87 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:26.327800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753185f-8ae5-445c-a443-b911d6efd8e4 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:26.328355 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753185f-8ae5-445c-a443-b911d6efd8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:26.328779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753185f-8ae5-445c-a443-b911d6efd8e4 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:26.330571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 344/691] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:26.335370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16d614-4479-45e5-9c47-0b2a7fc0b479 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:26.335862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16d614-4479-45e5-9c47-0b2a7fc0b479 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:26.336375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16d614-4479-45e5-9c47-0b2a7fc0b479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:26.357004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4e16d614-4479-45e5-9c47-0b2a7fc0b479 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:26.358444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4e16d614-4479-45e5-9c47-0b2a7fc0b479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:26.392689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:26.412470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eef98e-3fc8-4b87-b730-4c06c4103a87 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:26.439593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 345/692] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:21:26.454076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9350e77-0d17-4dec-bde4-3937be9cc41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:26.455392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9350e77-0d17-4dec-bde4-3937be9cc41e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:26.455825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9350e77-0d17-4dec-bde4-3937be9cc41e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:26.456331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9350e77-0d17-4dec-bde4-3937be9cc41e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:26.465609 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16d614-4479-45e5-9c47-0b2a7fc0b479 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:26.474536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9350e77-0d17-4dec-bde4-3937be9cc41e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:26.477049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9350e77-0d17-4dec-bde4-3937be9cc41e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-d9350e77-0d17-4dec-bde4-3937be9cc41e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:26.506534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:26.507489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:26.508401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:26.508401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:26.508401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:26.508401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:26.509709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 348/693] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:26.515075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9350e77-0d17-4dec-bde4-3937be9cc41e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:26.553403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:26.567459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16d614-4479-45e5-9c47-0b2a7fc0b479 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:26.603655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5f4c7-2528-4c6f-8d9c-8c3b8d7c8c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:26.604559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5f4c7-2528-4c6f-8d9c-8c3b8d7c8c82 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:26.604985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5f4c7-2528-4c6f-8d9c-8c3b8d7c8c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:26.605360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5f4c7-2528-4c6f-8d9c-8c3b8d7c8c82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:26.617069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5f4c7-2528-4c6f-8d9c-8c3b8d7c8c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-9ab5f4c7-2528-4c6f-8d9c-8c3b8d7c8c82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:26.625377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:26.626324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:26.627172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:26.628702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9350e77-0d17-4dec-bde4-3937be9cc41e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:26.628848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 346/694] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:26.632438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5f4c7-2528-4c6f-8d9c-8c3b8d7c8c82 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:26.663420 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c067-6d7b-457d-81d5-d6890b584c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:26.664139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c067-6d7b-457d-81d5-d6890b584c07 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:26.664461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c067-6d7b-457d-81d5-d6890b584c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:26.664829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c067-6d7b-457d-81d5-d6890b584c07 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:26.665483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:26.665705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6c067-6d7b-457d-81d5-d6890b584c07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:26.665983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6c067-6d7b-457d-81d5-d6890b584c07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:26.667814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6c067-6d7b-457d-81d5-d6890b584c07 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:26.684609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 349/695] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:26.686508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c067-6d7b-457d-81d5-d6890b584c07 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:26.738600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5f4c7-2528-4c6f-8d9c-8c3b8d7c8c82 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:26.775494 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4bbba-dd3b-4d48-b0fb-a76958a0529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:26.776169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4bbba-dd3b-4d48-b0fb-a76958a0529a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:26.776481 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4bbba-dd3b-4d48-b0fb-a76958a0529a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:26.776762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4bbba-dd3b-4d48-b0fb-a76958a0529a None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:26.777523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4bbba-dd3b-4d48-b0fb-a76958a0529a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:26.777835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4bbba-dd3b-4d48-b0fb-a76958a0529a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:26.779447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4bbba-dd3b-4d48-b0fb-a76958a0529a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:26.796825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:26.797528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 347/696] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:26.803794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c067-6d7b-457d-81d5-d6890b584c07 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:26.804282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4bbba-dd3b-4d48-b0fb-a76958a0529a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:26.843589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78785846-8c3b-47c1-befd-73ea0a19b246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:26.844475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78785846-8c3b-47c1-befd-73ea0a19b246 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:26.844761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78785846-8c3b-47c1-befd-73ea0a19b246 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:26.845084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78785846-8c3b-47c1-befd-73ea0a19b246 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:26.852178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78785846-8c3b-47c1-befd-73ea0a19b246 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:26.853874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78785846-8c3b-47c1-befd-73ea0a19b246 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-78785846-8c3b-47c1-befd-73ea0a19b246 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:26.868835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:26.869784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:26.870661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:26.870661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:26.870661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:26.870661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:26.870661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:26.870661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:26.870661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:26.870661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:26.871085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 350/697] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:26.876344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78785846-8c3b-47c1-befd-73ea0a19b246 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:26.980764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4bbba-dd3b-4d48-b0fb-a76958a0529a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:26.988741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:27.004407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78785846-8c3b-47c1-befd-73ea0a19b246 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:27.017620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecfc20-e429-42ed-a415-840f73c2c6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:27.018440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecfc20-e429-42ed-a415-840f73c2c6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:27.018794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecfc20-e429-42ed-a415-840f73c2c6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:27.019075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecfc20-e429-42ed-a415-840f73c2c6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:27.031299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecfc20-e429-42ed-a415-840f73c2c6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:27.033002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecfc20-e429-42ed-a415-840f73c2c6f4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:27.041419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f5749-e640-49a1-91f0-0ee737542d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:27.042180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f5749-e640-49a1-91f0-0ee737542d3a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:27.042470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f5749-e640-49a1-91f0-0ee737542d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:27.042748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f5749-e640-49a1-91f0-0ee737542d3a None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:27.043606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f5749-e640-49a1-91f0-0ee737542d3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:27.043920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f5749-e640-49a1-91f0-0ee737542d3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:27.045426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f5749-e640-49a1-91f0-0ee737542d3a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-04ecfc20-e429-42ed-a415-840f73c2c6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:27.051682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:27.052566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:27.053455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:27.053455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:27.053455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:27.053455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:27.053455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 348/698] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:27.058469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 351/699] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:27.059183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecfc20-e429-42ed-a415-840f73c2c6f4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:27.064081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f5749-e640-49a1-91f0-0ee737542d3a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:27.081014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:27.168113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecfc20-e429-42ed-a415-840f73c2c6f4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:27.171335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f5749-e640-49a1-91f0-0ee737542d3a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:27.203770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5ac24-a796-4ee2-934f-e6c835b6fe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:27.204547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5ac24-a796-4ee2-934f-e6c835b6fe5f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:27.205360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5ac24-a796-4ee2-934f-e6c835b6fe5f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:27.205645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5ac24-a796-4ee2-934f-e6c835b6fe5f None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:27.207761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5ac24-a796-4ee2-934f-e6c835b6fe5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:27.208092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5ac24-a796-4ee2-934f-e6c835b6fe5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:27.209519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5ac24-a796-4ee2-934f-e6c835b6fe5f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:27.209721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52594a4a-e441-40d1-ae89-f737a62cf55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:27.210553 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52594a4a-e441-40d1-ae89-f737a62cf55d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:27.211188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52594a4a-e441-40d1-ae89-f737a62cf55d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:27.211652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52594a4a-e441-40d1-ae89-f737a62cf55d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:27.213270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52594a4a-e441-40d1-ae89-f737a62cf55d None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=8a26412cf9f64b1bbbd8e595b55bcebc) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:27.213764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52594a4a-e441-40d1-ae89-f737a62cf55d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:27.215351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52594a4a-e441-40d1-ae89-f737a62cf55d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:27.222087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 352/700] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:27 2022] GET /identity/v3/role_assignments?user.id=8a26412cf9f64b1bbbd8e595b55bcebc&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:27.233117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:27.237118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 349/701] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:27.243496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52594a4a-e441-40d1-ae89-f737a62cf55d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:27.380705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:27.416057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52594a4a-e441-40d1-ae89-f737a62cf55d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:27.453199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5289b8-0d6d-48d0-b8bd-1eae9241cc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:27.453999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5289b8-0d6d-48d0-b8bd-1eae9241cc9f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:27.454280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5289b8-0d6d-48d0-b8bd-1eae9241cc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:27.454550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5289b8-0d6d-48d0-b8bd-1eae9241cc9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:27.463261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5289b8-0d6d-48d0-b8bd-1eae9241cc9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:27.464752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5289b8-0d6d-48d0-b8bd-1eae9241cc9f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8e5289b8-0d6d-48d0-b8bd-1eae9241cc9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:27.479241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:27.480190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:27.481184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 353/702] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jul 21 17:21:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:27.486302 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5ac24-a796-4ee2-934f-e6c835b6fe5f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:27.591856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5ac24-a796-4ee2-934f-e6c835b6fe5f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:27.628533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f725c0-0322-4d7a-bac3-cf23e88769dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:27.629189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f725c0-0322-4d7a-bac3-cf23e88769dc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:27.629464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f725c0-0322-4d7a-bac3-cf23e88769dc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:27.629735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f725c0-0322-4d7a-bac3-cf23e88769dc None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:27.630566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f725c0-0322-4d7a-bac3-cf23e88769dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:27.630923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f725c0-0322-4d7a-bac3-cf23e88769dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:27.632309 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f725c0-0322-4d7a-bac3-cf23e88769dc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:27.650800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 350/703] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jul 21 17:21:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:27.656229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5289b8-0d6d-48d0-b8bd-1eae9241cc9f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:27.669248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7628e18-d413-425a-be1a-8fd5d3440d20 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:27.669537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7628e18-d413-425a-be1a-8fd5d3440d20 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:27.669789 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7628e18-d413-425a-be1a-8fd5d3440d20 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:27.671480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 351/704] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:27.677250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad441-4e8b-4a42-b2fb-c0162609dfb2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:27.677782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad441-4e8b-4a42-b2fb-c0162609dfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:27.678221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad441-4e8b-4a42-b2fb-c0162609dfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:27.703914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7dbad441-4e8b-4a42-b2fb-c0162609dfb2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:27.705447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7dbad441-4e8b-4a42-b2fb-c0162609dfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:27.773945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5289b8-0d6d-48d0-b8bd-1eae9241cc9f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:27.786102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:27.791421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 352/705] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:27.811938 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa57877-4123-4c94-8562-4f363f830d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:27.812637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa57877-4123-4c94-8562-4f363f830d05 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:27.812924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa57877-4123-4c94-8562-4f363f830d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:27.813203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa57877-4123-4c94-8562-4f363f830d05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:27.815890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbad441-4e8b-4a42-b2fb-c0162609dfb2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:27.823319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:27.905148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa57877-4123-4c94-8562-4f363f830d05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:27.906544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa57877-4123-4c94-8562-4f363f830d05 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:27.925247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbad441-4e8b-4a42-b2fb-c0162609dfb2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:27.977618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959aa582-acab-4f13-a2aa-2aecd6195503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:27.978786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959aa582-acab-4f13-a2aa-2aecd6195503 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:27.979212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959aa582-acab-4f13-a2aa-2aecd6195503 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:27.979652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959aa582-acab-4f13-a2aa-2aecd6195503 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:27.980491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959aa582-acab-4f13-a2aa-2aecd6195503 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:27.982020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959aa582-acab-4f13-a2aa-2aecd6195503 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-959aa582-acab-4f13-a2aa-2aecd6195503 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:28.015255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:28.016515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 17:21:28.017570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:28.020038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 353/706] 10.222.0.57 () {56 vars in 1129 bytes} [Thu Jul 21 17:21:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:28.029197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959aa582-acab-4f13-a2aa-2aecd6195503 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:28.031389 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 354/707] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:28.039382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa57877-4123-4c94-8562-4f363f830d05 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:28.100741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:28.123298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:28.208216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa57877-4123-4c94-8562-4f363f830d05 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:28.226547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959aa582-acab-4f13-a2aa-2aecd6195503 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:28.252366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ab2da-2af2-46af-9f71-7bce27eaccb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:28.253217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ab2da-2af2-46af-9f71-7bce27eaccb8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:28.253648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ab2da-2af2-46af-9f71-7bce27eaccb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:28.254075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ab2da-2af2-46af-9f71-7bce27eaccb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:28.265908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ab2da-2af2-46af-9f71-7bce27eaccb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:28.267384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ab2da-2af2-46af-9f71-7bce27eaccb8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a77ab2da-2af2-46af-9f71-7bce27eaccb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:28.286312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:28.287255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:28.288199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:28.288199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:28.288199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:28.288199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:28.288199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 355/708] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:28.297246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ab2da-2af2-46af-9f71-7bce27eaccb8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:28.303624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160d0c5-6fa6-442c-864c-ee91db634ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:28.304323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160d0c5-6fa6-442c-864c-ee91db634ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:28.304597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160d0c5-6fa6-442c-864c-ee91db634ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:28.304868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160d0c5-6fa6-442c-864c-ee91db634ff1 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:28.305579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160d0c5-6fa6-442c-864c-ee91db634ff1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:28.306073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160d0c5-6fa6-442c-864c-ee91db634ff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:28.307533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160d0c5-6fa6-442c-864c-ee91db634ff1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:28.312573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 354/709] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jul 21 17:21:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:28.317841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160d0c5-6fa6-442c-864c-ee91db634ff1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:28.420789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:28.431987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ab2da-2af2-46af-9f71-7bce27eaccb8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:28.474371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160d0c5-6fa6-442c-864c-ee91db634ff1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:28.477202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b75370-caef-464f-b558-cdcfd93b0b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:28.477952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b75370-caef-464f-b558-cdcfd93b0b40 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:28.478365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b75370-caef-464f-b558-cdcfd93b0b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:28.478655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b75370-caef-464f-b558-cdcfd93b0b40 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:28.479393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b75370-caef-464f-b558-cdcfd93b0b40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:28.479704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b75370-caef-464f-b558-cdcfd93b0b40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:28.481104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b75370-caef-464f-b558-cdcfd93b0b40 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:28.493241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 356/710] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:28.498452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b75370-caef-464f-b558-cdcfd93b0b40 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:28.503804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:28.543028 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d6941-1d44-454e-b42c-9b39cc2d2b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:28.543705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d6941-1d44-454e-b42c-9b39cc2d2b65 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:28.544497 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d6941-1d44-454e-b42c-9b39cc2d2b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:28.544784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d6941-1d44-454e-b42c-9b39cc2d2b65 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:28.545662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d6941-1d44-454e-b42c-9b39cc2d2b65 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:28.546100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d6941-1d44-454e-b42c-9b39cc2d2b65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:28.547676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d6941-1d44-454e-b42c-9b39cc2d2b65 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:28.556195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 355/711] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jul 21 17:21:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:28.559738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a72cd-9604-4841-b1ca-6b2733ff7850 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:28.560176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a72cd-9604-4841-b1ca-6b2733ff7850 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:28.560637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a72cd-9604-4841-b1ca-6b2733ff7850 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:28.563220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 356/712] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:28.569312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba172-081e-4a6f-8543-edbf989680cc None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:28.569601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba172-081e-4a6f-8543-edbf989680cc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:28.570727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba172-081e-4a6f-8543-edbf989680cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:28.613576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-62cba172-081e-4a6f-8543-edbf989680cc None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:28.614828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-62cba172-081e-4a6f-8543-edbf989680cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:28.718391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b75370-caef-464f-b558-cdcfd93b0b40 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:28.750942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 357/713] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:21:28.781463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba172-081e-4a6f-8543-edbf989680cc None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:28.782627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b75196-e16a-4287-9dd5-d91a44eecb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:28.783445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b75196-e16a-4287-9dd5-d91a44eecb4d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:28.783728 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b75196-e16a-4287-9dd5-d91a44eecb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:28.784005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b75196-e16a-4287-9dd5-d91a44eecb4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:28.799216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b75196-e16a-4287-9dd5-d91a44eecb4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:28.799216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b75196-e16a-4287-9dd5-d91a44eecb4d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-85b75196-e16a-4287-9dd5-d91a44eecb4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:28.829141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:28.830385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:28.831353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:28.831353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:28.831353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:28.831353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:28.831353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:28.831353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:28.831353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:28.831353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:28.833427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 357/714] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:28.834936 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18c98a-a9fc-4f68-9818-971213ae6992 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:28.835436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18c98a-a9fc-4f68-9818-971213ae6992 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:28.840347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18c98a-a9fc-4f68-9818-971213ae6992 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:28.842467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 358/715] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:28.843830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18c98a-a9fc-4f68-9818-971213ae6992 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:28.860736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:28.942444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba172-081e-4a6f-8543-edbf989680cc None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:28.958052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:28.989316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80793c4-4081-46af-b4f4-87f2e39c1703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:28.990466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80793c4-4081-46af-b4f4-87f2e39c1703 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:28.990923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80793c4-4081-46af-b4f4-87f2e39c1703 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:28.991053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18c98a-a9fc-4f68-9818-971213ae6992 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:28.991822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80793c4-4081-46af-b4f4-87f2e39c1703 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:29.003864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80793c4-4081-46af-b4f4-87f2e39c1703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-d80793c4-4081-46af-b4f4-87f2e39c1703 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:29.012550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:29.013717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:29.014791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:29.016920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 358/716] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:29.023011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85293100-4679-4c20-b7bb-c3a478ccf142 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:29.023514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85293100-4679-4c20-b7bb-c3a478ccf142 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:29.023994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85293100-4679-4c20-b7bb-c3a478ccf142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:29.037198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e472987-cf7f-4f3f-ae5d-92223e625be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:29.037869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e472987-cf7f-4f3f-ae5d-92223e625be4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:29.038149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e472987-cf7f-4f3f-ae5d-92223e625be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:29.038593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e472987-cf7f-4f3f-ae5d-92223e625be4 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:29.039535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e472987-cf7f-4f3f-ae5d-92223e625be4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:29.039896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e472987-cf7f-4f3f-ae5d-92223e625be4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:29.042251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e472987-cf7f-4f3f-ae5d-92223e625be4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:29.043288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-85293100-4679-4c20-b7bb-c3a478ccf142 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:29.044710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-85293100-4679-4c20-b7bb-c3a478ccf142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:29.064749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 359/717] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:29.064749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e472987-cf7f-4f3f-ae5d-92223e625be4 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:29.148198 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 359/718] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:29.149119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85293100-4679-4c20-b7bb-c3a478ccf142 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:29.153485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:29.200718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e472987-cf7f-4f3f-ae5d-92223e625be4 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:29.240152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b04e04-5293-404b-bf4e-3e5e441ef79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:29.240876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b04e04-5293-404b-bf4e-3e5e441ef79f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:29.241160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b04e04-5293-404b-bf4e-3e5e441ef79f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:29.241436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b04e04-5293-404b-bf4e-3e5e441ef79f None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:29.242191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b04e04-5293-404b-bf4e-3e5e441ef79f None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:29.242495 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b04e04-5293-404b-bf4e-3e5e441ef79f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:29.244465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b04e04-5293-404b-bf4e-3e5e441ef79f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:29.254077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 360/719] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:29.254887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b04e04-5293-404b-bf4e-3e5e441ef79f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:29.286612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:29.290942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85293100-4679-4c20-b7bb-c3a478ccf142 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:29.341182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbe32b-1893-4fce-a4fa-63c15039e1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:29.342179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbe32b-1893-4fce-a4fa-63c15039e1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:29.342653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbe32b-1893-4fce-a4fa-63c15039e1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:29.343100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbe32b-1893-4fce-a4fa-63c15039e1b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:29.344103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbe32b-1893-4fce-a4fa-63c15039e1b7 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=380d6fe7057d44bfb20b0ae2bb1e664f) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:29.344610 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbe32b-1893-4fce-a4fa-63c15039e1b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:29.346743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbe32b-1893-4fce-a4fa-63c15039e1b7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:29.355204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 360/720] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:29 2022] GET /identity/v3/role_assignments?user.id=380d6fe7057d44bfb20b0ae2bb1e664f&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:29.356488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbe32b-1893-4fce-a4fa-63c15039e1b7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:29.382322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b04e04-5293-404b-bf4e-3e5e441ef79f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:29.426937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:29.447425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3fbcc-d34e-4182-9da9-f71939483197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:29.448607 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3fbcc-d34e-4182-9da9-f71939483197 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:29.449325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3fbcc-d34e-4182-9da9-f71939483197 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:29.449775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3fbcc-d34e-4182-9da9-f71939483197 None admin] PATH_INFO: `/v3/services/network` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:29.451452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3fbcc-d34e-4182-9da9-f71939483197 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:29.453699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3fbcc-d34e-4182-9da9-f71939483197 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:29.490025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbe32b-1893-4fce-a4fa-63c15039e1b7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6ab3fbcc-d34e-4182-9da9-f71939483197 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:29.494080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:29.495068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 17:21:29.495911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:29.504870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 361/721] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:29 2022] GET /identity/v3/services/network => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:29.511957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3fbcc-d34e-4182-9da9-f71939483197 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:29.547504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de12c3-a507-4fe8-b618-f17458a638ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:29.548490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de12c3-a507-4fe8-b618-f17458a638ca None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:29.548800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de12c3-a507-4fe8-b618-f17458a638ca None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:29.549091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de12c3-a507-4fe8-b618-f17458a638ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:29.569813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de12c3-a507-4fe8-b618-f17458a638ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:29.571210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de12c3-a507-4fe8-b618-f17458a638ca None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f9de12c3-a507-4fe8-b618-f17458a638ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:29.598118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:29.599247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:29.600243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:29.600243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:29.600243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:29.600243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:29.602233 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 361/722] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:29.609036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de12c3-a507-4fe8-b618-f17458a638ca None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:29.630901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:29.666676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3fbcc-d34e-4182-9da9-f71939483197 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:29.706579 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dba9f7-a0cd-4569-9ded-a07c4c21cfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:29.707251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dba9f7-a0cd-4569-9ded-a07c4c21cfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:29.707538 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dba9f7-a0cd-4569-9ded-a07c4c21cfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:29.707827 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dba9f7-a0cd-4569-9ded-a07c4c21cfd1 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:29.708557 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dba9f7-a0cd-4569-9ded-a07c4c21cfd1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:29.708863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dba9f7-a0cd-4569-9ded-a07c4c21cfd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:29.710251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dba9f7-a0cd-4569-9ded-a07c4c21cfd1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:29.723707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 362/723] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:21:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:29.735121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dba9f7-a0cd-4569-9ded-a07c4c21cfd1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:29.787144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de12c3-a507-4fe8-b618-f17458a638ca None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:29.805117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:29.848919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0ad88-f4a6-4141-b12b-7de87b7d2ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:29.849609 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0ad88-f4a6-4141-b12b-7de87b7d2ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:29.849895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0ad88-f4a6-4141-b12b-7de87b7d2ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:29.850172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0ad88-f4a6-4141-b12b-7de87b7d2ec1 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:29.850909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0ad88-f4a6-4141-b12b-7de87b7d2ec1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:29.851220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0ad88-f4a6-4141-b12b-7de87b7d2ec1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:29.852853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0ad88-f4a6-4141-b12b-7de87b7d2ec1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:29.885470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 362/724] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:29.888964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0ad88-f4a6-4141-b12b-7de87b7d2ec1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:29.893385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dba9f7-a0cd-4569-9ded-a07c4c21cfd1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:29.964903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280831c-cbee-4047-9c3e-74de814eac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:29.965577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280831c-cbee-4047-9c3e-74de814eac90 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:29.965857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280831c-cbee-4047-9c3e-74de814eac90 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:29.966132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280831c-cbee-4047-9c3e-74de814eac90 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:29.966846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280831c-cbee-4047-9c3e-74de814eac90 None admin] RBAC: Adding query filter params (type=network) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:29.967151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280831c-cbee-4047-9c3e-74de814eac90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:29.968697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280831c-cbee-4047-9c3e-74de814eac90 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:29.980326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 363/725] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:21:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:30.027407 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:30.059331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0ad88-f4a6-4141-b12b-7de87b7d2ec1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:30.115536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb69c9-043e-43e3-9871-ccda7411c2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:30.116403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb69c9-043e-43e3-9871-ccda7411c2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:30.116691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb69c9-043e-43e3-9871-ccda7411c2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:30.116973 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb69c9-043e-43e3-9871-ccda7411c2c5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:30.125415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb69c9-043e-43e3-9871-ccda7411c2c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:30.126912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb69c9-043e-43e3-9871-ccda7411c2c5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-7fbb69c9-043e-43e3-9871-ccda7411c2c5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:30.152948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:30.153943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:30.154869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 363/726] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jul 21 17:21:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:30.160090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280831c-cbee-4047-9c3e-74de814eac90 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:30.163986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaffc8a-149e-43b6-8727-d31c1dd7c9fd None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:30.164277 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaffc8a-149e-43b6-8727-d31c1dd7c9fd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:30.164581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaffc8a-149e-43b6-8727-d31c1dd7c9fd None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:30.166767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 364/727] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:30.171480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b001d4-b12b-42ef-9c15-10a309002c3a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:30.171770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b001d4-b12b-42ef-9c15-10a309002c3a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:30.172050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b001d4-b12b-42ef-9c15-10a309002c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:30.194416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-05b001d4-b12b-42ef-9c15-10a309002c3a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:30.195614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-05b001d4-b12b-42ef-9c15-10a309002c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:30.271687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 365/728] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 17:21:30.282361 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280831c-cbee-4047-9c3e-74de814eac90 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:30.293979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b001d4-b12b-42ef-9c15-10a309002c3a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:30.338363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:30.361980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160f75b-846e-4660-9986-55887ff204b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:30.362677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160f75b-846e-4660-9986-55887ff204b1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:30.363322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160f75b-846e-4660-9986-55887ff204b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:30.363605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160f75b-846e-4660-9986-55887ff204b1 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:30.364494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160f75b-846e-4660-9986-55887ff204b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:30.364811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160f75b-846e-4660-9986-55887ff204b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:30.366568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160f75b-846e-4660-9986-55887ff204b1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:30.398374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:30.404452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 364/729] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jul 21 17:21:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:30.406115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160f75b-846e-4660-9986-55887ff204b1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:30.436803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b001d4-b12b-42ef-9c15-10a309002c3a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:30.485514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84b85b-5600-4fb2-bb05-3fbb69ad9344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:30.486663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84b85b-5600-4fb2-bb05-3fbb69ad9344 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:30.487139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84b85b-5600-4fb2-bb05-3fbb69ad9344 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:30.487604 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84b85b-5600-4fb2-bb05-3fbb69ad9344 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:30.508511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84b85b-5600-4fb2-bb05-3fbb69ad9344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:30.520331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160f75b-846e-4660-9986-55887ff204b1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6e84b85b-5600-4fb2-bb05-3fbb69ad9344 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:30.520504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:30.521374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:30.522212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 366/730] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:30.530734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84b85b-5600-4fb2-bb05-3fbb69ad9344 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:30.580094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9bb1c7-06b0-4236-9062-18a24d147f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:30.580784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bb1c7-06b0-4236-9062-18a24d147f64 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:30.581067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bb1c7-06b0-4236-9062-18a24d147f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:30.581342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bb1c7-06b0-4236-9062-18a24d147f64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:30.675359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:30.703307 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9bb1c7-06b0-4236-9062-18a24d147f64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:30.704945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9bb1c7-06b0-4236-9062-18a24d147f64 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:30.707291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84b85b-5600-4fb2-bb05-3fbb69ad9344 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:30.737033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:30.758390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a134eea-2eb4-4ee4-bbf2-74ec1ae876f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:30.759574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a134eea-2eb4-4ee4-bbf2-74ec1ae876f8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:30.760110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a134eea-2eb4-4ee4-bbf2-74ec1ae876f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:30.760593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a134eea-2eb4-4ee4-bbf2-74ec1ae876f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:30.761538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a134eea-2eb4-4ee4-bbf2-74ec1ae876f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:30.762041 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a134eea-2eb4-4ee4-bbf2-74ec1ae876f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:30.763652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a134eea-2eb4-4ee4-bbf2-74ec1ae876f8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:30.779752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 367/731] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:30.785386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a134eea-2eb4-4ee4-bbf2-74ec1ae876f8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:30.840526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 365/732] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 17:21:30.841799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9bb1c7-06b0-4236-9062-18a24d147f64 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:30.942229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:30.963393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a134eea-2eb4-4ee4-bbf2-74ec1ae876f8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:30.968279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9bb1c7-06b0-4236-9062-18a24d147f64 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:31.000599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d21cc-ac25-42c2-9556-fed04966110f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:31.001813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d21cc-ac25-42c2-9556-fed04966110f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.002191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d21cc-ac25-42c2-9556-fed04966110f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.002562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d21cc-ac25-42c2-9556-fed04966110f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.013480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57b370-3547-4ec7-8b0f-eb23f4a1a2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:31.014753 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b370-3547-4ec7-8b0f-eb23f4a1a2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.015185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b370-3547-4ec7-8b0f-eb23f4a1a2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.015560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d21cc-ac25-42c2-9556-fed04966110f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:31.015784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b370-3547-4ec7-8b0f-eb23f4a1a2f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.017234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d21cc-ac25-42c2-9556-fed04966110f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:31.029272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57b370-3547-4ec7-8b0f-eb23f4a1a2f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:31.030880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57b370-3547-4ec7-8b0f-eb23f4a1a2f4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-fc2d21cc-ac25-42c2-9556-fed04966110f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:31.037870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:31.038786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:31.039684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:31.039684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:31.039684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:31.039684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:31.039684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 368/733] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:31.046836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af99425-bfae-46b1-bb03-44653afa9202 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.047127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af99425-bfae-46b1-bb03-44653afa9202 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.047512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af99425-bfae-46b1-bb03-44653afa9202 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.049196 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 369/734] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:31.050235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af99425-bfae-46b1-bb03-44653afa9202 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-ba57b370-3547-4ec7-8b0f-eb23f4a1a2f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:31.057367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:31.058270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:31.059190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:31.059190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:31.059190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:31.059190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:31.066055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 366/735] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:31.066055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098edab-f41d-4390-b4e5-58dfbe004820 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.066055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098edab-f41d-4390-b4e5-58dfbe004820 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.066055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098edab-f41d-4390-b4e5-58dfbe004820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.093104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5098edab-f41d-4390-b4e5-58dfbe004820 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:31.094403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5098edab-f41d-4390-b4e5-58dfbe004820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:31.108262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:31.171490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 367/736] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:31.172970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098edab-f41d-4390-b4e5-58dfbe004820 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:31.174819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af99425-bfae-46b1-bb03-44653afa9202 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:31.221335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5756cc-ffa6-499a-aae9-9f4ccc4ad485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:31.222475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5756cc-ffa6-499a-aae9-9f4ccc4ad485 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.222902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5756cc-ffa6-499a-aae9-9f4ccc4ad485 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.223452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5756cc-ffa6-499a-aae9-9f4ccc4ad485 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.224743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5756cc-ffa6-499a-aae9-9f4ccc4ad485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:31.225198 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5756cc-ffa6-499a-aae9-9f4ccc4ad485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:31.227097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5756cc-ffa6-499a-aae9-9f4ccc4ad485 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:31.242202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 370/737] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:31.247278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5756cc-ffa6-499a-aae9-9f4ccc4ad485 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:31.249375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:31.289455 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098edab-f41d-4390-b4e5-58dfbe004820 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:31.328145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1dcac7-c943-4a29-8ca5-c5feb022e72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:31.329032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dcac7-c943-4a29-8ca5-c5feb022e72f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.329319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dcac7-c943-4a29-8ca5-c5feb022e72f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.329599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dcac7-c943-4a29-8ca5-c5feb022e72f None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.330717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1dcac7-c943-4a29-8ca5-c5feb022e72f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:31.331058 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1dcac7-c943-4a29-8ca5-c5feb022e72f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:31.332931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1dcac7-c943-4a29-8ca5-c5feb022e72f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:31.348011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 368/738] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:31.349013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1dcac7-c943-4a29-8ca5-c5feb022e72f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:31.377953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:31.382981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5756cc-ffa6-499a-aae9-9f4ccc4ad485 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:31.426484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ae90a-0f85-42ff-9166-281bce82195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:31.427148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ae90a-0f85-42ff-9166-281bce82195a None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.427431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ae90a-0f85-42ff-9166-281bce82195a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.427706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ae90a-0f85-42ff-9166-281bce82195a None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.428459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ae90a-0f85-42ff-9166-281bce82195a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:31.429823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ae90a-0f85-42ff-9166-281bce82195a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:31.450523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 371/739] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:21:31 2022] POST /identity/v3/services => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:31.451424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ae90a-0f85-42ff-9166-281bce82195a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:31.491248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1dcac7-c943-4a29-8ca5-c5feb022e72f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:31.526224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:31.545500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c8d38-5917-4527-8222-1eb79fc60b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:31.546324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c8d38-5917-4527-8222-1eb79fc60b8a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.546605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c8d38-5917-4527-8222-1eb79fc60b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.546879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c8d38-5917-4527-8222-1eb79fc60b8a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.555121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c8d38-5917-4527-8222-1eb79fc60b8a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:31.556711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c8d38-5917-4527-8222-1eb79fc60b8a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-2d6c8d38-5917-4527-8222-1eb79fc60b8a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:31.579741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:31.580814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:31.581743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 369/740] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:31.589218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c8d38-5917-4527-8222-1eb79fc60b8a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:31.603094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ae90a-0f85-42ff-9166-281bce82195a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:31.680407 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e495f-b9a5-4358-8067-6e120bba7ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:31.681565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e495f-b9a5-4358-8067-6e120bba7ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.682049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e495f-b9a5-4358-8067-6e120bba7ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.682501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e495f-b9a5-4358-8067-6e120bba7ed8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.693912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e495f-b9a5-4358-8067-6e120bba7ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:31.695956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e495f-b9a5-4358-8067-6e120bba7ed8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-bb6e495f-b9a5-4358-8067-6e120bba7ed8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:31.713840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:31.715210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:31.716098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:31.716098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:31.716098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:31.716098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:31.716098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:31.716098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:31.716098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:31.716098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:31.720459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 372/741] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:31.722237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e495f-b9a5-4358-8067-6e120bba7ed8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:31.750951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:31.770207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c8d38-5917-4527-8222-1eb79fc60b8a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:31.809155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7d77c-97ec-4743-9ade-bd5472e3ecd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:31.809860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d77c-97ec-4743-9ade-bd5472e3ecd2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.810150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d77c-97ec-4743-9ade-bd5472e3ecd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.810422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d77c-97ec-4743-9ade-bd5472e3ecd2 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.811390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7d77c-97ec-4743-9ade-bd5472e3ecd2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:31.811698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7d77c-97ec-4743-9ade-bd5472e3ecd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:31.813293 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7d77c-97ec-4743-9ade-bd5472e3ecd2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:31.836739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 370/742] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jul 21 17:21:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:31.839062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f86220-ccb1-49e9-b3bb-95ff01d2018f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.839554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f86220-ccb1-49e9-b3bb-95ff01d2018f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.840032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f86220-ccb1-49e9-b3bb-95ff01d2018f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.841824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 371/743] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:31.843019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f86220-ccb1-49e9-b3bb-95ff01d2018f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:31.845393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e495f-b9a5-4358-8067-6e120bba7ed8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:31.882798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198ee16-ce35-4f40-a6c3-aec0dd208bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:31.882798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198ee16-ce35-4f40-a6c3-aec0dd208bad None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.883073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198ee16-ce35-4f40-a6c3-aec0dd208bad None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.883305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198ee16-ce35-4f40-a6c3-aec0dd208bad None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.884127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198ee16-ce35-4f40-a6c3-aec0dd208bad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:31.884462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198ee16-ce35-4f40-a6c3-aec0dd208bad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:31.885899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198ee16-ce35-4f40-a6c3-aec0dd208bad None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:31.889475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:31.901646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 373/744] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:31.903405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e1261-0474-4ae0-a144-7cf0d66d1be7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.903996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e1261-0474-4ae0-a144-7cf0d66d1be7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.904507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e1261-0474-4ae0-a144-7cf0d66d1be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:31.933856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-462e1261-0474-4ae0-a144-7cf0d66d1be7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:31.935550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-462e1261-0474-4ae0-a144-7cf0d66d1be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:31.959131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f86220-ccb1-49e9-b3bb-95ff01d2018f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:31.998334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600b4c9-7588-498f-9a1b-46408e850f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:31.998993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600b4c9-7588-498f-9a1b-46408e850f4d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:31.999277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600b4c9-7588-498f-9a1b-46408e850f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:31.999554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600b4c9-7588-498f-9a1b-46408e850f4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:32.012616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 374/745] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:32.013840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e1261-0474-4ae0-a144-7cf0d66d1be7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:32.026188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:32.078313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600b4c9-7588-498f-9a1b-46408e850f4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:32.079973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600b4c9-7588-498f-9a1b-46408e850f4d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:32.122354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:32.124176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e1261-0474-4ae0-a144-7cf0d66d1be7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:32.161757 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47fac5-fe15-4af8-a990-6dcb087a5681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:32.162753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47fac5-fe15-4af8-a990-6dcb087a5681 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:32.163176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47fac5-fe15-4af8-a990-6dcb087a5681 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:32.163623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47fac5-fe15-4af8-a990-6dcb087a5681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:32.164722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47fac5-fe15-4af8-a990-6dcb087a5681 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=75f11eaa4b504d0aa4018fb4ffbbd3a2) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:32.165706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47fac5-fe15-4af8-a990-6dcb087a5681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:32.167401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47fac5-fe15-4af8-a990-6dcb087a5681 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:32.173958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 375/746] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:32 2022] GET /identity/v3/role_assignments?user.id=75f11eaa4b504d0aa4018fb4ffbbd3a2&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:32.175113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47fac5-fe15-4af8-a990-6dcb087a5681 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:32.197107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 372/747] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 17:21:32.199161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600b4c9-7588-498f-9a1b-46408e850f4d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:32.268815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:32.293064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47fac5-fe15-4af8-a990-6dcb087a5681 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:32.331985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5388-a446-4f6a-83e0-7e9fc31f4d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:32.333157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5388-a446-4f6a-83e0-7e9fc31f4d14 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:32.333615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5388-a446-4f6a-83e0-7e9fc31f4d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:32.334062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5388-a446-4f6a-83e0-7e9fc31f4d14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:32.346214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b5388-a446-4f6a-83e0-7e9fc31f4d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-025b5388-a446-4f6a-83e0-7e9fc31f4d14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:32.355392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:32.356417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:32.357267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:32.360120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 376/748] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:32.366103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5388-a446-4f6a-83e0-7e9fc31f4d14 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:32.495694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600b4c9-7588-498f-9a1b-46408e850f4d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:32.526551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5388-a446-4f6a-83e0-7e9fc31f4d14 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:32.561725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664def3-d16f-4c20-a625-22e0c3b15acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:32.563474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664def3-d16f-4c20-a625-22e0c3b15acb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:32.564183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664def3-d16f-4c20-a625-22e0c3b15acb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:32.564685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664def3-d16f-4c20-a625-22e0c3b15acb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:32.576391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:32.578144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ab70c-cfaf-4146-afb9-7e44a48fc4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:32.578953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ab70c-cfaf-4146-afb9-7e44a48fc4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:32.579236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ab70c-cfaf-4146-afb9-7e44a48fc4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:32.579510 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ab70c-cfaf-4146-afb9-7e44a48fc4a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:32.580313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ab70c-cfaf-4146-afb9-7e44a48fc4a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:32.580621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ab70c-cfaf-4146-afb9-7e44a48fc4a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:32.581988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ab70c-cfaf-4146-afb9-7e44a48fc4a1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:32.583499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:32.587948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664def3-d16f-4c20-a625-22e0c3b15acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:32.589454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664def3-d16f-4c20-a625-22e0c3b15acb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:32.595914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 377/749] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:32.603400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5592c-cf45-40e6-ad8e-fdf295571858 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:32.603854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5592c-cf45-40e6-ad8e-fdf295571858 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:32.604417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5592c-cf45-40e6-ad8e-fdf295571858 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:32.606685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 378/750] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:32.607556 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5592c-cf45-40e6-ad8e-fdf295571858 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-0664def3-d16f-4c20-a625-22e0c3b15acb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:32.614957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:32.615853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:32.616811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:32.616811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:32.616811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:32.616811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:32.616811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 373/751] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:32.623441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69051d8-5a8c-45ee-a249-e422ccf7ec26 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:32.623802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69051d8-5a8c-45ee-a249-e422ccf7ec26 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:32.624153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69051d8-5a8c-45ee-a249-e422ccf7ec26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:32.644744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b69051d8-5a8c-45ee-a249-e422ccf7ec26 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:32.646467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b69051d8-5a8c-45ee-a249-e422ccf7ec26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:32.736322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 374/752] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 17:21:32.736322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69051d8-5a8c-45ee-a249-e422ccf7ec26 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:32.752235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5592c-cf45-40e6-ad8e-fdf295571858 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:32.788387 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043e02e-ced1-4c63-a4a2-bfde2ea15d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:32.789197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043e02e-ced1-4c63-a4a2-bfde2ea15d59 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:32.789489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043e02e-ced1-4c63-a4a2-bfde2ea15d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:32.789765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043e02e-ced1-4c63-a4a2-bfde2ea15d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:32.802576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043e02e-ced1-4c63-a4a2-bfde2ea15d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:32.804182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043e02e-ced1-4c63-a4a2-bfde2ea15d59 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-8043e02e-ced1-4c63-a4a2-bfde2ea15d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:32.823383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:32.824654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:32.825579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:32.825579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:32.825579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:32.825579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:32.825579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 379/753] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:32.825867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043e02e-ced1-4c63-a4a2-bfde2ea15d59 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:32.853623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69051d8-5a8c-45ee-a249-e422ccf7ec26 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:32.875542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:32.880457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:32.894331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a4dd6-929e-4904-9045-0e9344106815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:32.895007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a4dd6-929e-4904-9045-0e9344106815 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:32.895357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a4dd6-929e-4904-9045-0e9344106815 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:32.895699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a4dd6-929e-4904-9045-0e9344106815 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:32.896599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a4dd6-929e-4904-9045-0e9344106815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:32.896973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a4dd6-929e-4904-9045-0e9344106815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:32.898579 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a4dd6-929e-4904-9045-0e9344106815 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:32.916160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 375/754] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:32.916160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a4dd6-929e-4904-9045-0e9344106815 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:32.957243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043e02e-ced1-4c63-a4a2-bfde2ea15d59 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:32.994193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102b4ca-859a-41df-a7a1-0e54c00dda6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:32.995280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102b4ca-859a-41df-a7a1-0e54c00dda6f None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:32.995739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102b4ca-859a-41df-a7a1-0e54c00dda6f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:32.996182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102b4ca-859a-41df-a7a1-0e54c00dda6f None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:32.997234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102b4ca-859a-41df-a7a1-0e54c00dda6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:32.998966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102b4ca-859a-41df-a7a1-0e54c00dda6f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:33.017024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 380/755] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:21:32 2022] POST /identity/v3/services => generated 240 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:33.018409 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102b4ca-859a-41df-a7a1-0e54c00dda6f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:33.032402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a4dd6-929e-4904-9045-0e9344106815 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:33.106462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25c02a-d612-4c46-8f98-028791d4593c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:33.107215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25c02a-d612-4c46-8f98-028791d4593c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:33.107609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25c02a-d612-4c46-8f98-028791d4593c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:33.107953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25c02a-d612-4c46-8f98-028791d4593c None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:33.108776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25c02a-d612-4c46-8f98-028791d4593c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:33.109166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25c02a-d612-4c46-8f98-028791d4593c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:33.110652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25c02a-d612-4c46-8f98-028791d4593c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:33.136380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 376/756] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:33.141946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25c02a-d612-4c46-8f98-028791d4593c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:33.142383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102b4ca-859a-41df-a7a1-0e54c00dda6f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:33.153909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:33.190096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dacfaa-cbde-4197-a6ce-c842e2b5ab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:33.191061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dacfaa-cbde-4197-a6ce-c842e2b5ab8a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:33.191441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dacfaa-cbde-4197-a6ce-c842e2b5ab8a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:33.192106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dacfaa-cbde-4197-a6ce-c842e2b5ab8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:33.201808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dacfaa-cbde-4197-a6ce-c842e2b5ab8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:33.203815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dacfaa-cbde-4197-a6ce-c842e2b5ab8a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:33.211279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-33dacfaa-cbde-4197-a6ce-c842e2b5ab8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:33.224829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:33.225871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:33.226752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:33.226752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:33.226752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:33.226752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:33.226752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:33.226752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:33.226752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:33.226752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:33.231447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 381/757] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:33.237742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dacfaa-cbde-4197-a6ce-c842e2b5ab8a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:33.357282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25c02a-d612-4c46-8f98-028791d4593c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:33.377853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dacfaa-cbde-4197-a6ce-c842e2b5ab8a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:33.396782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5ca44-57b9-43c9-bd97-ec03818f427e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:33.397586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5ca44-57b9-43c9-bd97-ec03818f427e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:33.397865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5ca44-57b9-43c9-bd97-ec03818f427e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:33.398134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5ca44-57b9-43c9-bd97-ec03818f427e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:33.410252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5ca44-57b9-43c9-bd97-ec03818f427e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:33.411812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5ca44-57b9-43c9-bd97-ec03818f427e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:33.423331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328f1ea-2486-4c78-b2de-0ba67e62f2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:33.424002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f1ea-2486-4c78-b2de-0ba67e62f2ce None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:33.424281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f1ea-2486-4c78-b2de-0ba67e62f2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:33.424580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f1ea-2486-4c78-b2de-0ba67e62f2ce None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:33.425378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328f1ea-2486-4c78-b2de-0ba67e62f2ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:33.425687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328f1ea-2486-4c78-b2de-0ba67e62f2ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:33.427124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328f1ea-2486-4c78-b2de-0ba67e62f2ce None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f3b5ca44-57b9-43c9-bd97-ec03818f427e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:33.429343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:33.430287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:33.431318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:33.431318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:33.431318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:33.431318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:33.431318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:33.431318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:33.431318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:33.431318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:33.431318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:33.431318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:33.431867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 377/758] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jul 21 17:21:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:33.437550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5ca44-57b9-43c9-bd97-ec03818f427e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:33.444992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 382/759] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:33.455873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328f1ea-2486-4c78-b2de-0ba67e62f2ce None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:33.498556 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:33.564680 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:33.573015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5ca44-57b9-43c9-bd97-ec03818f427e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:33.610934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328f1ea-2486-4c78-b2de-0ba67e62f2ce None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:33.646249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98ca9b-2847-48c3-8378-a118d591253d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:33.646902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98ca9b-2847-48c3-8378-a118d591253d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:33.647182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98ca9b-2847-48c3-8378-a118d591253d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:33.647456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98ca9b-2847-48c3-8378-a118d591253d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:33.648285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98ca9b-2847-48c3-8378-a118d591253d None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=8a26412cf9f64b1bbbd8e595b55bcebc) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:33.648637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98ca9b-2847-48c3-8378-a118d591253d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:33.650336 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98ca9b-2847-48c3-8378-a118d591253d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:33.650633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024e2ba-fd17-4998-8af9-608e801dcb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:33.651537 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024e2ba-fd17-4998-8af9-608e801dcb9a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:33.652032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024e2ba-fd17-4998-8af9-608e801dcb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:33.652533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024e2ba-fd17-4998-8af9-608e801dcb9a None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:33.653590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024e2ba-fd17-4998-8af9-608e801dcb9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:33.654099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024e2ba-fd17-4998-8af9-608e801dcb9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:33.664340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024e2ba-fd17-4998-8af9-608e801dcb9a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:33.664543 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 383/760] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:33 2022] GET /identity/v3/role_assignments?user.id=8a26412cf9f64b1bbbd8e595b55bcebc&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:33.676647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 378/761] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jul 21 17:21:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:33.685424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98ca9b-2847-48c3-8378-a118d591253d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:33.789688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:33.800738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98ca9b-2847-48c3-8378-a118d591253d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:33.844195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ddefe-b09d-42aa-830f-3d44008aa913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:33.845031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ddefe-b09d-42aa-830f-3d44008aa913 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:33.845309 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ddefe-b09d-42aa-830f-3d44008aa913 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:33.845579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ddefe-b09d-42aa-830f-3d44008aa913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:33.858002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ddefe-b09d-42aa-830f-3d44008aa913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:33.859464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ddefe-b09d-42aa-830f-3d44008aa913 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-467ddefe-b09d-42aa-830f-3d44008aa913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:33.883383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:33.884719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:33.885736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:33.885736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:33.885736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:33.885736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:33.885736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 384/762] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:33.890644 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024e2ba-fd17-4998-8af9-608e801dcb9a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:33.901535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaa1ab-aefb-4ee8-bee9-9a083fea2cca None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:33.901847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaa1ab-aefb-4ee8-bee9-9a083fea2cca None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:33.902068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaa1ab-aefb-4ee8-bee9-9a083fea2cca None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:33.906113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 385/763] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:33.909834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c5e4a-abe1-4104-812c-d6c70c3f040b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:33.910124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c5e4a-abe1-4104-812c-d6c70c3f040b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:33.910403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c5e4a-abe1-4104-812c-d6c70c3f040b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:33.941505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2f1c5e4a-abe1-4104-812c-d6c70c3f040b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:33.942817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2f1c5e4a-abe1-4104-812c-d6c70c3f040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:34.016887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024e2ba-fd17-4998-8af9-608e801dcb9a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:34.056452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a639077-dbd1-4f69-af9a-dbab4f53d06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:34.057633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a639077-dbd1-4f69-af9a-dbab4f53d06e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:34.058120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a639077-dbd1-4f69-af9a-dbab4f53d06e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:34.058597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a639077-dbd1-4f69-af9a-dbab4f53d06e None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:34.059525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a639077-dbd1-4f69-af9a-dbab4f53d06e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:34.060037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a639077-dbd1-4f69-af9a-dbab4f53d06e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:34.061837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a639077-dbd1-4f69-af9a-dbab4f53d06e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:34.082963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 379/764] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:34.089363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a639077-dbd1-4f69-af9a-dbab4f53d06e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:34.089907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 386/765] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:34.112970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c5e4a-abe1-4104-812c-d6c70c3f040b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:34.119322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:34.196757 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:34.229032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c5e4a-abe1-4104-812c-d6c70c3f040b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:34.273009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df213ae-9d0d-4227-a7ee-cc10f574b2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:34.273828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df213ae-9d0d-4227-a7ee-cc10f574b2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:34.274112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df213ae-9d0d-4227-a7ee-cc10f574b2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:34.274421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df213ae-9d0d-4227-a7ee-cc10f574b2a5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:34.275083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df213ae-9d0d-4227-a7ee-cc10f574b2a5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:34.276495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df213ae-9d0d-4227-a7ee-cc10f574b2a5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-1df213ae-9d0d-4227-a7ee-cc10f574b2a5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:34.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:34.306817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:34.307651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 387/766] 10.222.0.57 () {56 vars in 1129 bytes} [Thu Jul 21 17:21:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:34.309126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a639077-dbd1-4f69-af9a-dbab4f53d06e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:34.313492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df213ae-9d0d-4227-a7ee-cc10f574b2a5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:34.426184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df213ae-9d0d-4227-a7ee-cc10f574b2a5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:34.457113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba4af-549f-4c9f-a49e-6405d86f5d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:34.457860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba4af-549f-4c9f-a49e-6405d86f5d1f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:34.458148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba4af-549f-4c9f-a49e-6405d86f5d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:34.458427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba4af-549f-4c9f-a49e-6405d86f5d1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:34.463207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e66743-7752-45cc-8612-bd74e2161c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:34.464242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e66743-7752-45cc-8612-bd74e2161c27 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:34.464582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e66743-7752-45cc-8612-bd74e2161c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:34.464869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e66743-7752-45cc-8612-bd74e2161c27 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:34.465597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e66743-7752-45cc-8612-bd74e2161c27 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:34.465913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e66743-7752-45cc-8612-bd74e2161c27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:34.467355 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e66743-7752-45cc-8612-bd74e2161c27 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:34.476281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 388/767] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jul 21 17:21:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:34.477106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba4af-549f-4c9f-a49e-6405d86f5d1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:34.478251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e66743-7752-45cc-8612-bd74e2161c27 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:34.479133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba4af-549f-4c9f-a49e-6405d86f5d1f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:34.492659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-7f8ba4af-549f-4c9f-a49e-6405d86f5d1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:34.517487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:34.518415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:34.519300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:34.519300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:34.519300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:34.519300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:34.519300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:34.519300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:34.519300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:34.519300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:34.520453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 380/768] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:34.534169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba4af-549f-4c9f-a49e-6405d86f5d1f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:34.647692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:34.651533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e66743-7752-45cc-8612-bd74e2161c27 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:34.676994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba4af-549f-4c9f-a49e-6405d86f5d1f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:34.700505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ffcc6-160e-4008-9542-b7935a1a4de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:34.701186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ffcc6-160e-4008-9542-b7935a1a4de7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:34.701472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ffcc6-160e-4008-9542-b7935a1a4de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:34.701747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ffcc6-160e-4008-9542-b7935a1a4de7 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:34.702462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ffcc6-160e-4008-9542-b7935a1a4de7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:34.702900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ffcc6-160e-4008-9542-b7935a1a4de7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:34.704411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ffcc6-160e-4008-9542-b7935a1a4de7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:34.710851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 389/769] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jul 21 17:21:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:34.714086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc23bbe-99db-41ce-a655-f7048cf595c6 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:34.714380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc23bbe-99db-41ce-a655-f7048cf595c6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:34.714657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc23bbe-99db-41ce-a655-f7048cf595c6 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:34.716317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 390/770] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:34.718090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc23bbe-99db-41ce-a655-f7048cf595c6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:34.729593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59604ede-6bfe-4b4b-82f2-76336899c0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:34.730261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59604ede-6bfe-4b4b-82f2-76336899c0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:34.730546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59604ede-6bfe-4b4b-82f2-76336899c0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:34.730823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59604ede-6bfe-4b4b-82f2-76336899c0b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:34.731621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59604ede-6bfe-4b4b-82f2-76336899c0b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:34.731928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59604ede-6bfe-4b4b-82f2-76336899c0b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:34.733396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59604ede-6bfe-4b4b-82f2-76336899c0b4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:34.751801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 381/771] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:34.753240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b3330-cb43-4df2-b444-2dc48d50b563 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:34.753531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b3330-cb43-4df2-b444-2dc48d50b563 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:34.753815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b3330-cb43-4df2-b444-2dc48d50b563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:34.772177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-817b3330-cb43-4df2-b444-2dc48d50b563 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:34.773501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-817b3330-cb43-4df2-b444-2dc48d50b563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:34.790175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:34.831783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc23bbe-99db-41ce-a655-f7048cf595c6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:34.852473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 382/772] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:34.853463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b3330-cb43-4df2-b444-2dc48d50b563 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:34.867513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b11d51-df98-497b-99f6-c9cdda764754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:34.869356 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b11d51-df98-497b-99f6-c9cdda764754 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:34.869648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b11d51-df98-497b-99f6-c9cdda764754 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:34.869926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b11d51-df98-497b-99f6-c9cdda764754 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:34.870715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b11d51-df98-497b-99f6-c9cdda764754 None admin] RBAC: Adding query filter params (interface=public, service_id=9af5ec06bb514c729707c2fcb072dffd, region_id=RegionOne) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:34.871030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b11d51-df98-497b-99f6-c9cdda764754 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:34.872416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b11d51-df98-497b-99f6-c9cdda764754 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:34.880559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 391/773] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jul 21 17:21:34 2022] GET /identity/v3/endpoints?service_id=9af5ec06bb514c729707c2fcb072dffd&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:34.881374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b11d51-df98-497b-99f6-c9cdda764754 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:34.938948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:34.980892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b3330-cb43-4df2-b444-2dc48d50b563 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:34.996799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b11d51-df98-497b-99f6-c9cdda764754 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:35.017415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae710db-6cc5-4022-8da8-245f23a35a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:35.019109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae710db-6cc5-4022-8da8-245f23a35a2f None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:35.019580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae710db-6cc5-4022-8da8-245f23a35a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:35.020071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae710db-6cc5-4022-8da8-245f23a35a2f None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/380d6fe7057d44bfb20b0ae2bb1e664f/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:35.023401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae710db-6cc5-4022-8da8-245f23a35a2f None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=380d6fe7057d44bfb20b0ae2bb1e664f, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:35.025395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae710db-6cc5-4022-8da8-245f23a35a2f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:35.030105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:35.036615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94085762-12ff-4626-b96a-245defaa55de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:35.037515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94085762-12ff-4626-b96a-245defaa55de None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:35.037801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94085762-12ff-4626-b96a-245defaa55de None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:35.038072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94085762-12ff-4626-b96a-245defaa55de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:35.043408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 383/774] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:21:34 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/380d6fe7057d44bfb20b0ae2bb1e664f/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:21:35.044569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae710db-6cc5-4022-8da8-245f23a35a2f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:35.053536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94085762-12ff-4626-b96a-245defaa55de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-94085762-12ff-4626-b96a-245defaa55de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:35.063606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:35.064873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:35.065773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 392/775] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:35.070699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94085762-12ff-4626-b96a-245defaa55de None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:35.195010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94085762-12ff-4626-b96a-245defaa55de None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:35.207437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae710db-6cc5-4022-8da8-245f23a35a2f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:35.235814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497323d-31ee-44a7-925c-3ce5369ad79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:35.236996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497323d-31ee-44a7-925c-3ce5369ad79f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:35.237494 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497323d-31ee-44a7-925c-3ce5369ad79f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:35.238040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497323d-31ee-44a7-925c-3ce5369ad79f None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:35.239057 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497323d-31ee-44a7-925c-3ce5369ad79f None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:35.239560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497323d-31ee-44a7-925c-3ce5369ad79f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:35.241390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497323d-31ee-44a7-925c-3ce5369ad79f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:35.251223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 393/776] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:35.256657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497323d-31ee-44a7-925c-3ce5369ad79f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:35.260800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:35.288073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2d125-24e0-4bc1-9bed-787db41634bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:35.289025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2d125-24e0-4bc1-9bed-787db41634bc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:35.289208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2d125-24e0-4bc1-9bed-787db41634bc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:35.289443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2d125-24e0-4bc1-9bed-787db41634bc None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:35.290159 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2d125-24e0-4bc1-9bed-787db41634bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:35.291523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2d125-24e0-4bc1-9bed-787db41634bc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:35.297596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 384/777] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jul 21 17:21:35 2022] GET /identity/v3/services => generated 772 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:35.322943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:35.379484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497323d-31ee-44a7-925c-3ce5369ad79f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:35.417464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44edeed-47a6-4d2d-b7b2-06b934d1fa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:35.418542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44edeed-47a6-4d2d-b7b2-06b934d1fa7c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:35.418826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44edeed-47a6-4d2d-b7b2-06b934d1fa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:35.419103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44edeed-47a6-4d2d-b7b2-06b934d1fa7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:35.432396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44edeed-47a6-4d2d-b7b2-06b934d1fa7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:35.433877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44edeed-47a6-4d2d-b7b2-06b934d1fa7c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-a44edeed-47a6-4d2d-b7b2-06b934d1fa7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:35.453042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:35.453983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:35.454941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:35.454941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:35.454941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:35.454941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:35.454941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 394/778] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:35.461794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2d125-24e0-4bc1-9bed-787db41634bc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:35.575601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2d125-24e0-4bc1-9bed-787db41634bc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:35.618684 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977930dd-ef24-4423-b162-e65d945d43d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:35.619729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977930dd-ef24-4423-b162-e65d945d43d5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:35.620244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977930dd-ef24-4423-b162-e65d945d43d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:35.620733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977930dd-ef24-4423-b162-e65d945d43d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:35.621707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977930dd-ef24-4423-b162-e65d945d43d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:35.622220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977930dd-ef24-4423-b162-e65d945d43d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:35.623805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977930dd-ef24-4423-b162-e65d945d43d5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:35.644480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 385/779] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:35.649989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44edeed-47a6-4d2d-b7b2-06b934d1fa7c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:35.724894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453ae6f-cf82-45dd-a0e5-4368c5984e91 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:35.725522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453ae6f-cf82-45dd-a0e5-4368c5984e91 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:35.725996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453ae6f-cf82-45dd-a0e5-4368c5984e91 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:35.728066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 386/780] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:35.733665 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2d30a-3db4-4715-bbef-9014962a9c6d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:35.733946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2d30a-3db4-4715-bbef-9014962a9c6d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:35.734222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2d30a-3db4-4715-bbef-9014962a9c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:35.769350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-42e2d30a-3db4-4715-bbef-9014962a9c6d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:35.770558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-42e2d30a-3db4-4715-bbef-9014962a9c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:35.777802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44edeed-47a6-4d2d-b7b2-06b934d1fa7c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:35.781432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:35.843240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:35.848683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52fbd1-760e-4c17-b32b-94df6a4dcfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:35.851084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52fbd1-760e-4c17-b32b-94df6a4dcfea None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:35.851384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52fbd1-760e-4c17-b32b-94df6a4dcfea None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:35.851666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52fbd1-760e-4c17-b32b-94df6a4dcfea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:35.866397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52fbd1-760e-4c17-b32b-94df6a4dcfea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:35.868905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52fbd1-760e-4c17-b32b-94df6a4dcfea None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:35.888324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 387/781] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-fc52fbd1-760e-4c17-b32b-94df6a4dcfea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:35.893918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:35.894818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:35.895765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 395/782] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jul 21 17:21:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:35.902653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2d30a-3db4-4715-bbef-9014962a9c6d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:35.924388 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52fbd1-760e-4c17-b32b-94df6a4dcfea None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:36.043711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52fbd1-760e-4c17-b32b-94df6a4dcfea None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:36.079730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff62cf-cdb2-4dca-9c69-cb7675cab2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:36.080572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff62cf-cdb2-4dca-9c69-cb7675cab2ba None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.080856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff62cf-cdb2-4dca-9c69-cb7675cab2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.081127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff62cf-cdb2-4dca-9c69-cb7675cab2ba None admin] PATH_INFO: `/v3/services/network` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.081777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff62cf-cdb2-4dca-9c69-cb7675cab2ba None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:36.082226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2d30a-3db4-4715-bbef-9014962a9c6d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:36.083148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff62cf-cdb2-4dca-9c69-cb7675cab2ba None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-46ff62cf-cdb2-4dca-9c69-cb7675cab2ba None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:36.096866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:36.097896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:36.098783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 396/783] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:35 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:36.107985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff62cf-cdb2-4dca-9c69-cb7675cab2ba None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:36.154623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e66b1-2994-4e8b-90fa-d26372d4da96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:36.155307 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e66b1-2994-4e8b-90fa-d26372d4da96 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.155587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e66b1-2994-4e8b-90fa-d26372d4da96 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.155859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e66b1-2994-4e8b-90fa-d26372d4da96 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.156727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e66b1-2994-4e8b-90fa-d26372d4da96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:36.157035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e66b1-2994-4e8b-90fa-d26372d4da96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:36.158463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e66b1-2994-4e8b-90fa-d26372d4da96 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:36.166106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:36.197464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 388/784] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jul 21 17:21:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:36.199407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166e61-e4ac-4ba1-b627-e9f496424725 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.199853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166e61-e4ac-4ba1-b627-e9f496424725 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.200134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166e61-e4ac-4ba1-b627-e9f496424725 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.204386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 389/785] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:36.205168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166e61-e4ac-4ba1-b627-e9f496424725 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:36.255390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:36.258933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff62cf-cdb2-4dca-9c69-cb7675cab2ba None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:36.296264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23992b-b5ae-4aed-b1fc-c6c4cb801019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:36.297124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23992b-b5ae-4aed-b1fc-c6c4cb801019 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.297501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23992b-b5ae-4aed-b1fc-c6c4cb801019 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.297871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23992b-b5ae-4aed-b1fc-c6c4cb801019 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.298643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23992b-b5ae-4aed-b1fc-c6c4cb801019 None admin] RBAC: Adding query filter params (name=network) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:36.299020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23992b-b5ae-4aed-b1fc-c6c4cb801019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:36.300431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23992b-b5ae-4aed-b1fc-c6c4cb801019 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:36.305245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 397/786] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:21:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:36.306308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a10db-8079-471a-b8df-5a7dc13f1be9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.306590 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a10db-8079-471a-b8df-5a7dc13f1be9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.306979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a10db-8079-471a-b8df-5a7dc13f1be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.326338 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-428a10db-8079-471a-b8df-5a7dc13f1be9 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:36.327557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-428a10db-8079-471a-b8df-5a7dc13f1be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:36.349304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166e61-e4ac-4ba1-b627-e9f496424725 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:36.385954 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2afbe-c435-4d7b-b264-f39cc6c4be13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:36.386867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2afbe-c435-4d7b-b264-f39cc6c4be13 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.387209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2afbe-c435-4d7b-b264-f39cc6c4be13 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.387547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2afbe-c435-4d7b-b264-f39cc6c4be13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.402857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2afbe-c435-4d7b-b264-f39cc6c4be13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:36.404331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2afbe-c435-4d7b-b264-f39cc6c4be13 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:36.406125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 398/787] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:36.407011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a10db-8079-471a-b8df-5a7dc13f1be9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-9dc2afbe-c435-4d7b-b264-f39cc6c4be13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:36.438448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:36.439397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:36.440334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:36.440334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:36.440334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:36.440334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:36.442674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 390/788] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:36.443692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2afbe-c435-4d7b-b264-f39cc6c4be13 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:36.448442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:36.531963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a10db-8079-471a-b8df-5a7dc13f1be9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:36.546696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:36.561762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2afbe-c435-4d7b-b264-f39cc6c4be13 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:36.576621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4baa7-07eb-4c20-b04a-c0d09630755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:36.577709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4baa7-07eb-4c20-b04a-c0d09630755f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.578280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4baa7-07eb-4c20-b04a-c0d09630755f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.578742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4baa7-07eb-4c20-b04a-c0d09630755f None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.579672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4baa7-07eb-4c20-b04a-c0d09630755f None admin] RBAC: Adding query filter params (type=network) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:36.580168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4baa7-07eb-4c20-b04a-c0d09630755f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:36.581919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4baa7-07eb-4c20-b04a-c0d09630755f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:36.588729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 399/789] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:21:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:36.589895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4baa7-07eb-4c20-b04a-c0d09630755f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:36.609884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845f1ce-4ffe-4619-a343-7032e7366dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:36.610846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845f1ce-4ffe-4619-a343-7032e7366dce None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.611203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845f1ce-4ffe-4619-a343-7032e7366dce None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.611554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845f1ce-4ffe-4619-a343-7032e7366dce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.623428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845f1ce-4ffe-4619-a343-7032e7366dce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8845f1ce-4ffe-4619-a343-7032e7366dce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:36.631915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:36.633002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:36.634542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 391/790] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:36.635244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845f1ce-4ffe-4619-a343-7032e7366dce None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:36.691093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:36.710543 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4baa7-07eb-4c20-b04a-c0d09630755f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:36.754469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa7f33-b2f7-4267-8157-8b4d8168fa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:36.756003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa7f33-b2f7-4267-8157-8b4d8168fa08 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.756482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa7f33-b2f7-4267-8157-8b4d8168fa08 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.756934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa7f33-b2f7-4267-8157-8b4d8168fa08 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.758877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa7f33-b2f7-4267-8157-8b4d8168fa08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:36.759379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa7f33-b2f7-4267-8157-8b4d8168fa08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:36.760176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845f1ce-4ffe-4619-a343-7032e7366dce None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:36.761598 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa7f33-b2f7-4267-8157-8b4d8168fa08 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:36.783173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 400/791] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:36.785189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa7f33-b2f7-4267-8157-8b4d8168fa08 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:36.806596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655a6ed-0251-472a-bd45-04bceacb0b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:36.807659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655a6ed-0251-472a-bd45-04bceacb0b0d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.808143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655a6ed-0251-472a-bd45-04bceacb0b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.808649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655a6ed-0251-472a-bd45-04bceacb0b0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.809668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655a6ed-0251-472a-bd45-04bceacb0b0d None admin] RBAC: Adding query filter params (interface=public, service_id=342f23aebb524d919a784513eb4030bb, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:36.810180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655a6ed-0251-472a-bd45-04bceacb0b0d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:36.811800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655a6ed-0251-472a-bd45-04bceacb0b0d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:36.820181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 392/792] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jul 21 17:21:36 2022] GET /identity/v3/endpoints?service_id=342f23aebb524d919a784513eb4030bb&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:36.822075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655a6ed-0251-472a-bd45-04bceacb0b0d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:36.842707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:36.913393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa7f33-b2f7-4267-8157-8b4d8168fa08 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:36.955426 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf89ef-6092-4611-a016-55db21110ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:36.956775 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf89ef-6092-4611-a016-55db21110ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:36.957260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf89ef-6092-4611-a016-55db21110ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:36.957732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf89ef-6092-4611-a016-55db21110ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:36.958692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf89ef-6092-4611-a016-55db21110ff5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:36.959260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf89ef-6092-4611-a016-55db21110ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:36.960960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf89ef-6092-4611-a016-55db21110ff5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:36.970483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655a6ed-0251-472a-bd45-04bceacb0b0d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:36.971278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 401/793] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:36.972441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf89ef-6092-4611-a016-55db21110ff5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:36.987792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:37.021648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cf25d-58bb-4363-9653-44e94b069640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:37.022457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cf25d-58bb-4363-9653-44e94b069640 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.022738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cf25d-58bb-4363-9653-44e94b069640 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.023012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cf25d-58bb-4363-9653-44e94b069640 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.032170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cf25d-58bb-4363-9653-44e94b069640 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:37.034306 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cf25d-58bb-4363-9653-44e94b069640 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-cd3cf25d-58bb-4363-9653-44e94b069640 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:37.053548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:37.054509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:37.056105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:37.056105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:37.056105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:37.056105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:37.056105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:37.056105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:37.056105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:37.056105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:37.060192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 393/794] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:37.062035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cf25d-58bb-4363-9653-44e94b069640 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:37.096566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf89ef-6092-4611-a016-55db21110ff5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:37.136358 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b10b0-6470-46b4-a584-45b6741d8fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:37.137475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b10b0-6470-46b4-a584-45b6741d8fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.137961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b10b0-6470-46b4-a584-45b6741d8fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.138487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b10b0-6470-46b4-a584-45b6741d8fc7 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.139372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b10b0-6470-46b4-a584-45b6741d8fc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:37.141024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b10b0-6470-46b4-a584-45b6741d8fc7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:37.147260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 402/795] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jul 21 17:21:36 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:37.148452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b10b0-6470-46b4-a584-45b6741d8fc7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:37.188523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:37.214989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cf25d-58bb-4363-9653-44e94b069640 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:37.287449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320145d-f5fc-4725-a281-fd41decd0e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:37.288737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320145d-f5fc-4725-a281-fd41decd0e75 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.289225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320145d-f5fc-4725-a281-fd41decd0e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.289744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320145d-f5fc-4725-a281-fd41decd0e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.308277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320145d-f5fc-4725-a281-fd41decd0e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:37.308519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:37.309796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320145d-f5fc-4725-a281-fd41decd0e75 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e320145d-f5fc-4725-a281-fd41decd0e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:37.342388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:37.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:37.344280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:37.344280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:37.344280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:37.344280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:37.349068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 394/796] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:37.352861 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320145d-f5fc-4725-a281-fd41decd0e75 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:37.365909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b10b0-6470-46b4-a584-45b6741d8fc7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:37.413807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d19b1-81a0-4d69-88e7-5761738588c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:37.414566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d19b1-81a0-4d69-88e7-5761738588c1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.415381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d19b1-81a0-4d69-88e7-5761738588c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.415658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d19b1-81a0-4d69-88e7-5761738588c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.416484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d19b1-81a0-4d69-88e7-5761738588c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:37.416797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d19b1-81a0-4d69-88e7-5761738588c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:37.418324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d19b1-81a0-4d69-88e7-5761738588c1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:37.432632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 403/797] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:37.438412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d19b1-81a0-4d69-88e7-5761738588c1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:37.468867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320145d-f5fc-4725-a281-fd41decd0e75 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:37.494579 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:37.506176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037924cc-4140-44c0-a47a-10038994210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:37.506844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037924cc-4140-44c0-a47a-10038994210d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.507125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037924cc-4140-44c0-a47a-10038994210d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.507435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037924cc-4140-44c0-a47a-10038994210d None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.508112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037924cc-4140-44c0-a47a-10038994210d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:37.509304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037924cc-4140-44c0-a47a-10038994210d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:37.510483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037924cc-4140-44c0-a47a-10038994210d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:37.528333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 395/798] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:37.528649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8bdf5-6b66-45cb-acaa-e088dd927a23 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.529858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8bdf5-6b66-45cb-acaa-e088dd927a23 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.529858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8bdf5-6b66-45cb-acaa-e088dd927a23 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.530978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 396/799] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:37.531246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8bdf5-6b66-45cb-acaa-e088dd927a23 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:37.568451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d19b1-81a0-4d69-88e7-5761738588c1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:37.601038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:37.607794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b29a25-e514-4ab7-80b1-cea8f1152277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:37.608811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29a25-e514-4ab7-80b1-cea8f1152277 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.609102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29a25-e514-4ab7-80b1-cea8f1152277 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.609383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29a25-e514-4ab7-80b1-cea8f1152277 None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/75f11eaa4b504d0aa4018fb4ffbbd3a2/roles/b9347810c7484efd95599fb9c51ab2df` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.625623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b29a25-e514-4ab7-80b1-cea8f1152277 None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=75f11eaa4b504d0aa4018fb4ffbbd3a2, role_id=b9347810c7484efd95599fb9c51ab2df)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:37.627129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b29a25-e514-4ab7-80b1-cea8f1152277 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:37.643116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 404/800] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:21:37 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/75f11eaa4b504d0aa4018fb4ffbbd3a2/roles/b9347810c7484efd95599fb9c51ab2df => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:21:37.645177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c15fd0-3464-4725-bdd0-a004f342c97f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.645500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c15fd0-3464-4725-bdd0-a004f342c97f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.645793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c15fd0-3464-4725-bdd0-a004f342c97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.671317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8bdf5-6b66-45cb-acaa-e088dd927a23 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:37.675766 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-55c15fd0-3464-4725-bdd0-a004f342c97f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:37.678337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-55c15fd0-3464-4725-bdd0-a004f342c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:37.716575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe689f0-2354-48dd-9f0c-8a7a4112ca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:37.717508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe689f0-2354-48dd-9f0c-8a7a4112ca4a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.717835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe689f0-2354-48dd-9f0c-8a7a4112ca4a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.718204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe689f0-2354-48dd-9f0c-8a7a4112ca4a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.728324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe689f0-2354-48dd-9f0c-8a7a4112ca4a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:37.729705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe689f0-2354-48dd-9f0c-8a7a4112ca4a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-5fe689f0-2354-48dd-9f0c-8a7a4112ca4a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:37.747275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:37.748320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:37.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:37.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:37.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:37.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:37.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:37.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:37.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:37.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:37.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:37.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:37.749805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 397/801] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:37.755937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69369d-6d5b-43a9-bde5-582395ccdf32 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.756244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69369d-6d5b-43a9-bde5-582395ccdf32 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.756510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69369d-6d5b-43a9-bde5-582395ccdf32 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.759121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 398/802] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:37.760558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69369d-6d5b-43a9-bde5-582395ccdf32 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:37.768869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 405/803] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:37.771033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b0124-0783-4aa7-a13f-d6d26d2dc0cd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.771321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b0124-0783-4aa7-a13f-d6d26d2dc0cd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.771598 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b0124-0783-4aa7-a13f-d6d26d2dc0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.794363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6b2b0124-0783-4aa7-a13f-d6d26d2dc0cd None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:37.795865 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6b2b0124-0783-4aa7-a13f-d6d26d2dc0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:37.821334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:37.879657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 406/804] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:37.880472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b0124-0783-4aa7-a13f-d6d26d2dc0cd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:37.902017 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69369d-6d5b-43a9-bde5-582395ccdf32 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:37.941461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:37.947268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c592e65-7a14-4a6e-82b1-7b86327e55d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:37.948537 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c592e65-7a14-4a6e-82b1-7b86327e55d2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:37.949123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c592e65-7a14-4a6e-82b1-7b86327e55d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:37.949499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c592e65-7a14-4a6e-82b1-7b86327e55d2 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:37.950833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c592e65-7a14-4a6e-82b1-7b86327e55d2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:37.951251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c592e65-7a14-4a6e-82b1-7b86327e55d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:37.952837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c592e65-7a14-4a6e-82b1-7b86327e55d2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:37.976824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 399/805] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jul 21 17:21:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:37.980439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c592e65-7a14-4a6e-82b1-7b86327e55d2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:38.014415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b0124-0783-4aa7-a13f-d6d26d2dc0cd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:38.053230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27e90e-fecb-454e-9a5b-1cfdbb33df2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:38.054411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27e90e-fecb-454e-9a5b-1cfdbb33df2d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:38.054842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27e90e-fecb-454e-9a5b-1cfdbb33df2d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:38.055378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27e90e-fecb-454e-9a5b-1cfdbb33df2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:38.067269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27e90e-fecb-454e-9a5b-1cfdbb33df2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-0d27e90e-fecb-454e-9a5b-1cfdbb33df2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:38.076126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:38.077241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:38.078126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:38.080991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 407/806] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:38.082087 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27e90e-fecb-454e-9a5b-1cfdbb33df2d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:38.113774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c592e65-7a14-4a6e-82b1-7b86327e55d2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:38.125805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:38.179968 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb44b6-c20a-41ec-be09-15aa3f72337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:38.180867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb44b6-c20a-41ec-be09-15aa3f72337f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:38.181148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb44b6-c20a-41ec-be09-15aa3f72337f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:38.181428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb44b6-c20a-41ec-be09-15aa3f72337f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:38.182095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb44b6-c20a-41ec-be09-15aa3f72337f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:38.183468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb44b6-c20a-41ec-be09-15aa3f72337f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1aeb44b6-c20a-41ec-be09-15aa3f72337f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:38.200989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:38.201989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 17:21:38.202832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:38.207443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 400/807] 10.222.0.57 () {56 vars in 1129 bytes} [Thu Jul 21 17:21:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:38.208334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb44b6-c20a-41ec-be09-15aa3f72337f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:38.220038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27e90e-fecb-454e-9a5b-1cfdbb33df2d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:38.222037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:38.262658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e570e-1b57-4c94-abb0-8882d277281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:38.263839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e570e-1b57-4c94-abb0-8882d277281b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:38.264344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e570e-1b57-4c94-abb0-8882d277281b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:38.264819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e570e-1b57-4c94-abb0-8882d277281b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:38.280845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e570e-1b57-4c94-abb0-8882d277281b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:38.282972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e570e-1b57-4c94-abb0-8882d277281b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ce3e570e-1b57-4c94-abb0-8882d277281b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:38.306330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:38.307293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:38.308190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:38.308190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:38.308190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:38.308190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:38.315367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 408/808] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:38.315367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e570e-1b57-4c94-abb0-8882d277281b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:38.362396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb44b6-c20a-41ec-be09-15aa3f72337f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:38.412574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9f583-1a87-4c6b-bc66-f3b303b57cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:38.413263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9f583-1a87-4c6b-bc66-f3b303b57cde None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:38.413550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9f583-1a87-4c6b-bc66-f3b303b57cde None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:38.413831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9f583-1a87-4c6b-bc66-f3b303b57cde None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:38.414633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9f583-1a87-4c6b-bc66-f3b303b57cde None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:38.414950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9f583-1a87-4c6b-bc66-f3b303b57cde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:38.416748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9f583-1a87-4c6b-bc66-f3b303b57cde None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:38.426180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 401/809] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:38.427327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9f583-1a87-4c6b-bc66-f3b303b57cde None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:38.452086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e570e-1b57-4c94-abb0-8882d277281b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:38.470887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:38.505169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ab7db-a492-46b2-a862-de7ff3068d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:38.505874 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ab7db-a492-46b2-a862-de7ff3068d1b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:38.506162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ab7db-a492-46b2-a862-de7ff3068d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:38.506459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ab7db-a492-46b2-a862-de7ff3068d1b None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:38.507183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ab7db-a492-46b2-a862-de7ff3068d1b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:38.507474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ab7db-a492-46b2-a862-de7ff3068d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:38.509670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ab7db-a492-46b2-a862-de7ff3068d1b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:38.520484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 409/810] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jul 21 17:21:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:38.521831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ab7db-a492-46b2-a862-de7ff3068d1b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:38.549755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:38.661919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ab7db-a492-46b2-a862-de7ff3068d1b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:38.705214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9f583-1a87-4c6b-bc66-f3b303b57cde None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:38.711130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a02ee-5410-452d-9a49-0434269fa1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:38.712337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a02ee-5410-452d-9a49-0434269fa1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:38.712831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a02ee-5410-452d-9a49-0434269fa1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:38.713323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a02ee-5410-452d-9a49-0434269fa1d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:38.728075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a02ee-5410-452d-9a49-0434269fa1d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:38.731312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a02ee-5410-452d-9a49-0434269fa1d8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:38.747291 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad4a5c-36c7-4de3-bcb6-07cc085e470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:38.747964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad4a5c-36c7-4de3-bcb6-07cc085e470b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:38.748247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad4a5c-36c7-4de3-bcb6-07cc085e470b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:38.748567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad4a5c-36c7-4de3-bcb6-07cc085e470b None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:38.749298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad4a5c-36c7-4de3-bcb6-07cc085e470b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:38.749607 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad4a5c-36c7-4de3-bcb6-07cc085e470b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:38.750967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad4a5c-36c7-4de3-bcb6-07cc085e470b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f52a02ee-5410-452d-9a49-0434269fa1d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:38.756116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:38.757036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:38.757941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:38.757941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:38.757941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:38.757941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:38.761935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 410/811] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:38.763270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a02ee-5410-452d-9a49-0434269fa1d8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:38.765014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 402/812] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:38.766951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad4a5c-36c7-4de3-bcb6-07cc085e470b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:38.858420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:38.876451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:38.892152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad4a5c-36c7-4de3-bcb6-07cc085e470b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:38.902325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a02ee-5410-452d-9a49-0434269fa1d8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:38.943711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72925ba-505f-4b67-b50d-7cf222d6629c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:38.945180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72925ba-505f-4b67-b50d-7cf222d6629c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:38.945632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72925ba-505f-4b67-b50d-7cf222d6629c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:38.946093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72925ba-505f-4b67-b50d-7cf222d6629c None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:38.947116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72925ba-505f-4b67-b50d-7cf222d6629c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:38.947464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72925ba-505f-4b67-b50d-7cf222d6629c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:38.949991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72925ba-505f-4b67-b50d-7cf222d6629c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:38.951926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11a528-116c-4cc5-815c-3f1027a9c1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:38.953441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11a528-116c-4cc5-815c-3f1027a9c1ef None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:38.954084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11a528-116c-4cc5-815c-3f1027a9c1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:38.954732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11a528-116c-4cc5-815c-3f1027a9c1ef None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:38.955657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11a528-116c-4cc5-815c-3f1027a9c1ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:38.956189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11a528-116c-4cc5-815c-3f1027a9c1ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:38.957930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11a528-116c-4cc5-815c-3f1027a9c1ef None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:38.963552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 403/813] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:38.964433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72925ba-505f-4b67-b50d-7cf222d6629c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:38.972193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 411/814] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Jul 21 17:21:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:38.974107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11a528-116c-4cc5-815c-3f1027a9c1ef None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:39.077186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72925ba-505f-4b67-b50d-7cf222d6629c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:39.105160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11a528-116c-4cc5-815c-3f1027a9c1ef None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:39.114932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e532deb-21f0-4a8b-a756-068286352af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:39.116020 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e532deb-21f0-4a8b-a756-068286352af8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:39.116456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e532deb-21f0-4a8b-a756-068286352af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:39.116904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e532deb-21f0-4a8b-a756-068286352af8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:39.125303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e532deb-21f0-4a8b-a756-068286352af8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:39.126887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e532deb-21f0-4a8b-a756-068286352af8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8e532deb-21f0-4a8b-a756-068286352af8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:39.141380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:39.142352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:39.143240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:39.143240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:39.143240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:39.143240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:39.143240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:39.143240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:39.143240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:39.143240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:39.143240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 404/815] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:39.143722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e532deb-21f0-4a8b-a756-068286352af8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:39.157559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ed949-9702-41fd-af4c-0bfa22329991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:39.158491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ed949-9702-41fd-af4c-0bfa22329991 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:39.158777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ed949-9702-41fd-af4c-0bfa22329991 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:39.159042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ed949-9702-41fd-af4c-0bfa22329991 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:39.160924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:39.162358 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:39.174032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ed949-9702-41fd-af4c-0bfa22329991 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:39.176346 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ed949-9702-41fd-af4c-0bfa22329991 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-3d8ed949-9702-41fd-af4c-0bfa22329991 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:39.197411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:39.199411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 412/816] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jul 21 17:21:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:39.202263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ed949-9702-41fd-af4c-0bfa22329991 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:39.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e532deb-21f0-4a8b-a756-068286352af8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:39.341181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ed949-9702-41fd-af4c-0bfa22329991 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:39.348723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7e420-5b1b-4fc9-b497-b79db5fc8836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:39.350081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e420-5b1b-4fc9-b497-b79db5fc8836 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:39.350547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e420-5b1b-4fc9-b497-b79db5fc8836 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:39.351002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e420-5b1b-4fc9-b497-b79db5fc8836 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:39.352467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7e420-5b1b-4fc9-b497-b79db5fc8836 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:39.354631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7e420-5b1b-4fc9-b497-b79db5fc8836 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:39.388746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 405/817] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jul 21 17:21:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:39.388746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7e420-5b1b-4fc9-b497-b79db5fc8836 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:39.397504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85775f6d-b6b7-4bbc-848b-f94e5afb13b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:39.399690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85775f6d-b6b7-4bbc-848b-f94e5afb13b4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:39.399963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85775f6d-b6b7-4bbc-848b-f94e5afb13b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:39.400583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85775f6d-b6b7-4bbc-848b-f94e5afb13b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:39.402495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85775f6d-b6b7-4bbc-848b-f94e5afb13b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:39.402842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85775f6d-b6b7-4bbc-848b-f94e5afb13b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:39.406572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85775f6d-b6b7-4bbc-848b-f94e5afb13b4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:39.436124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 413/818] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:39.441036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85775f6d-b6b7-4bbc-848b-f94e5afb13b4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:39.528329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:39.544251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:39.614870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7e420-5b1b-4fc9-b497-b79db5fc8836 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:39.627094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85775f6d-b6b7-4bbc-848b-f94e5afb13b4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:39.667526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630907b-81a8-4f04-a66a-11be0a0c75b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:39.668598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630907b-81a8-4f04-a66a-11be0a0c75b0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:39.669082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630907b-81a8-4f04-a66a-11be0a0c75b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:39.669540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630907b-81a8-4f04-a66a-11be0a0c75b0 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:39.670665 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630907b-81a8-4f04-a66a-11be0a0c75b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:39.671149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630907b-81a8-4f04-a66a-11be0a0c75b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:39.672927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630907b-81a8-4f04-a66a-11be0a0c75b0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:39.677574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628c85-6cc5-44a5-b61b-f752d8e5b07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:39.678153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628c85-6cc5-44a5-b61b-f752d8e5b07a None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:39.678501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628c85-6cc5-44a5-b61b-f752d8e5b07a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:39.678896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628c85-6cc5-44a5-b61b-f752d8e5b07a None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/8a26412cf9f64b1bbbd8e595b55bcebc/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:39.681077 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2628c85-6cc5-44a5-b61b-f752d8e5b07a None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=8a26412cf9f64b1bbbd8e595b55bcebc, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:39.682873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2628c85-6cc5-44a5-b61b-f752d8e5b07a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:39.698549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 406/819] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jul 21 17:21:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:39.702752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 414/820] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:21:39 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/8a26412cf9f64b1bbbd8e595b55bcebc/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:21:39.702874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bbcf3-a467-441e-957b-5a6f06515a00 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:39.703134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bbcf3-a467-441e-957b-5a6f06515a00 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:39.703550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bbcf3-a467-441e-957b-5a6f06515a00 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:39.704470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628c85-6cc5-44a5-b61b-f752d8e5b07a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:39.705047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 407/821] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:39.723931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7245dc-b811-4388-b315-6c6a120756dc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:39.724615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7245dc-b811-4388-b315-6c6a120756dc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:39.724615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7245dc-b811-4388-b315-6c6a120756dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:39.751012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-de7245dc-b811-4388-b315-6c6a120756dc None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:39.753023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-de7245dc-b811-4388-b315-6c6a120756dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:39.836265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628c85-6cc5-44a5-b61b-f752d8e5b07a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:39.852874 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:39.861303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 408/822] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:39.894994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e23190-c648-48b0-a218-0a0dc0ed8022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:39.895675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e23190-c648-48b0-a218-0a0dc0ed8022 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:39.895965 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e23190-c648-48b0-a218-0a0dc0ed8022 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:39.896247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e23190-c648-48b0-a218-0a0dc0ed8022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:39.911888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7245dc-b811-4388-b315-6c6a120756dc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:39.976012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e23190-c648-48b0-a218-0a0dc0ed8022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:39.977603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e23190-c648-48b0-a218-0a0dc0ed8022 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:39.980577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:40.040446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7245dc-b811-4388-b315-6c6a120756dc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:40.084813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe64c8-9c0b-4714-98fe-be9fbeeedd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:40.085759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe64c8-9c0b-4714-98fe-be9fbeeedd4d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.086044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe64c8-9c0b-4714-98fe-be9fbeeedd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.086315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe64c8-9c0b-4714-98fe-be9fbeeedd4d None admin] PATH_INFO: `/v3/services/network` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.087543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe64c8-9c0b-4714-98fe-be9fbeeedd4d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:40.089209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe64c8-9c0b-4714-98fe-be9fbeeedd4d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:40.100947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 415/823] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:21:40.102696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697291a-18c9-4b77-b68c-20a58a7c85ce None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-37fe64c8-9c0b-4714-98fe-be9fbeeedd4d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:40.102828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:40.103893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 17:21:40.105024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:40.106079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697291a-18c9-4b77-b68c-20a58a7c85ce None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.106079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697291a-18c9-4b77-b68c-20a58a7c85ce None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.106216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 409/824] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:39 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:40.106216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe64c8-9c0b-4714-98fe-be9fbeeedd4d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:40.106336 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 416/825] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:40.113748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d826b-1a01-492d-a6e8-01be9074e231 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.114040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d826b-1a01-492d-a6e8-01be9074e231 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.114323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d826b-1a01-492d-a6e8-01be9074e231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.141278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-313d826b-1a01-492d-a6e8-01be9074e231 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:40.142494 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-313d826b-1a01-492d-a6e8-01be9074e231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:40.145762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:40.219700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 417/826] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:40.220528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d826b-1a01-492d-a6e8-01be9074e231 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:40.232110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe64c8-9c0b-4714-98fe-be9fbeeedd4d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:40.250511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:40.270673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c9faa-a676-4c06-a5b9-cd4eb1571b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:40.271467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c9faa-a676-4c06-a5b9-cd4eb1571b48 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.271755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c9faa-a676-4c06-a5b9-cd4eb1571b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.272032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c9faa-a676-4c06-a5b9-cd4eb1571b48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.283690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c9faa-a676-4c06-a5b9-cd4eb1571b48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:40.285250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c9faa-a676-4c06-a5b9-cd4eb1571b48 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-fe0c9faa-a676-4c06-a5b9-cd4eb1571b48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:40.303653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:40.304833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:40.306403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:40.306403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:40.306403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:40.306403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:40.310370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 410/827] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:40.311353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c9faa-a676-4c06-a5b9-cd4eb1571b48 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:40.335420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d826b-1a01-492d-a6e8-01be9074e231 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:40.373570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5daa3cc-71eb-47e3-906b-56e7991313be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:40.374622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5daa3cc-71eb-47e3-906b-56e7991313be None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.375148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5daa3cc-71eb-47e3-906b-56e7991313be None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.375572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5daa3cc-71eb-47e3-906b-56e7991313be None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.376468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5daa3cc-71eb-47e3-906b-56e7991313be None admin] RBAC: Adding query filter params (name=network) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:40.376924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5daa3cc-71eb-47e3-906b-56e7991313be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:40.378639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5daa3cc-71eb-47e3-906b-56e7991313be None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:40.389327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 418/828] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:21:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:40.390412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5daa3cc-71eb-47e3-906b-56e7991313be None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:40.394486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:40.421818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c9faa-a676-4c06-a5b9-cd4eb1571b48 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:40.459853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e5d0d-75a3-4b41-82ec-0445a031c1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:40.461638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e5d0d-75a3-4b41-82ec-0445a031c1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.462103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e5d0d-75a3-4b41-82ec-0445a031c1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.462526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e5d0d-75a3-4b41-82ec-0445a031c1e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.476393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e5d0d-75a3-4b41-82ec-0445a031c1e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-295e5d0d-75a3-4b41-82ec-0445a031c1e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:40.487818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:40.488948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:21:40.489998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:40.493498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 411/829] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:21:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:40.494428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e5d0d-75a3-4b41-82ec-0445a031c1e5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:40.506518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5daa3cc-71eb-47e3-906b-56e7991313be None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:40.533576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:40.548266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231dfd5-8270-48dd-84dd-931d2783e0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:40.549342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231dfd5-8270-48dd-84dd-931d2783e0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.549808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231dfd5-8270-48dd-84dd-931d2783e0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.550232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231dfd5-8270-48dd-84dd-931d2783e0d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.551315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231dfd5-8270-48dd-84dd-931d2783e0d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:40.551770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231dfd5-8270-48dd-84dd-931d2783e0d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:40.553505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231dfd5-8270-48dd-84dd-931d2783e0d6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:40.567353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 419/830] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:40.568563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231dfd5-8270-48dd-84dd-931d2783e0d6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:40.634641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e5d0d-75a3-4b41-82ec-0445a031c1e5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:40.639148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:40.679514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b538383-c6c2-47b4-925a-108c7ffaf350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:40.679757 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231dfd5-8270-48dd-84dd-931d2783e0d6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:40.680749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b538383-c6c2-47b4-925a-108c7ffaf350 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.681036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b538383-c6c2-47b4-925a-108c7ffaf350 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.681315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b538383-c6c2-47b4-925a-108c7ffaf350 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.682066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b538383-c6c2-47b4-925a-108c7ffaf350 None admin] RBAC: Adding query filter params (type=network) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:40.682375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b538383-c6c2-47b4-925a-108c7ffaf350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:40.684350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b538383-c6c2-47b4-925a-108c7ffaf350 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:40.693356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 412/831] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:21:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:40.698012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b538383-c6c2-47b4-925a-108c7ffaf350 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:40.724038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35c3ae-0807-462f-864e-c053dd2b739b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:40.725098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35c3ae-0807-462f-864e-c053dd2b739b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.725553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35c3ae-0807-462f-864e-c053dd2b739b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.725981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35c3ae-0807-462f-864e-c053dd2b739b None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.727459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35c3ae-0807-462f-864e-c053dd2b739b None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:40.728077 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35c3ae-0807-462f-864e-c053dd2b739b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:40.729630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35c3ae-0807-462f-864e-c053dd2b739b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:40.739605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 420/832] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:21:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:40.740727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35c3ae-0807-462f-864e-c053dd2b739b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:40.808501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:40.823836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b538383-c6c2-47b4-925a-108c7ffaf350 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:40.854690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35c3ae-0807-462f-864e-c053dd2b739b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:40.871951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cab8c-64f2-4a1c-9b26-2bf68b7bc077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:40.873911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cab8c-64f2-4a1c-9b26-2bf68b7bc077 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.874453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cab8c-64f2-4a1c-9b26-2bf68b7bc077 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.875015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cab8c-64f2-4a1c-9b26-2bf68b7bc077 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.883457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:40.887281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cab8c-64f2-4a1c-9b26-2bf68b7bc077 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:40.888767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cab8c-64f2-4a1c-9b26-2bf68b7bc077 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:40.896542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb66c2e-c62e-4cc8-b2f8-bd8e907f5b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:40.897526 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb66c2e-c62e-4cc8-b2f8-bd8e907f5b50 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:40.898012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb66c2e-c62e-4cc8-b2f8-bd8e907f5b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:40.898491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb66c2e-c62e-4cc8-b2f8-bd8e907f5b50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:40.899998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb66c2e-c62e-4cc8-b2f8-bd8e907f5b50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:40.901590 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb66c2e-c62e-4cc8-b2f8-bd8e907f5b50 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-3c6cab8c-64f2-4a1c-9b26-2bf68b7bc077 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:40.904813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:40.905726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:40.906620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:40.906620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:40.906620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:40.906620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:40.906620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:40.906620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:40.906620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:40.906620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:40.911050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 413/833] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:40.911933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cab8c-64f2-4a1c-9b26-2bf68b7bc077 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:40.932809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 421/834] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jul 21 17:21:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:40.934374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb66c2e-c62e-4cc8-b2f8-bd8e907f5b50 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:41.104222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cab8c-64f2-4a1c-9b26-2bf68b7bc077 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:41.114255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb66c2e-c62e-4cc8-b2f8-bd8e907f5b50 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:41.144632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8a996-0674-4ef5-8c49-e38bf2051fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:41.145825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8a996-0674-4ef5-8c49-e38bf2051fcd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:41.146331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8a996-0674-4ef5-8c49-e38bf2051fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:41.146826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8a996-0674-4ef5-8c49-e38bf2051fcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:41.153686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7a36f-4437-4000-96a9-e3e1413adc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:41.154360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7a36f-4437-4000-96a9-e3e1413adc5e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:41.154643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7a36f-4437-4000-96a9-e3e1413adc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:41.154921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7a36f-4437-4000-96a9-e3e1413adc5e None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:41.155719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7a36f-4437-4000-96a9-e3e1413adc5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:41.156030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7a36f-4437-4000-96a9-e3e1413adc5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:41.157622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7a36f-4437-4000-96a9-e3e1413adc5e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:41.161135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8a996-0674-4ef5-8c49-e38bf2051fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:41.162796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8a996-0674-4ef5-8c49-e38bf2051fcd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:41.170819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 422/835] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:41.176635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7a36f-4437-4000-96a9-e3e1413adc5e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-b3d8a996-0674-4ef5-8c49-e38bf2051fcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:41.182357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:41.183257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:41.184190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:41.184190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:41.184190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:41.184190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:41.186464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 414/836] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:41.191647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8a996-0674-4ef5-8c49-e38bf2051fcd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:41.197435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:41.246890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:41.314199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8a996-0674-4ef5-8c49-e38bf2051fcd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:41.326093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7a36f-4437-4000-96a9-e3e1413adc5e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:41.352738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1cef1-f678-4207-a74e-b1fe1dec66d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:41.353292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1cef1-f678-4207-a74e-b1fe1dec66d6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:41.353648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1cef1-f678-4207-a74e-b1fe1dec66d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:41.354000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1cef1-f678-4207-a74e-b1fe1dec66d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:41.354961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1cef1-f678-4207-a74e-b1fe1dec66d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:41.355357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1cef1-f678-4207-a74e-b1fe1dec66d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:41.356904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1cef1-f678-4207-a74e-b1fe1dec66d6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:41.368172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06ff37-b937-4d34-ab48-68f74a9d7c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:41.368500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06ff37-b937-4d34-ab48-68f74a9d7c26 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:41.368781 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06ff37-b937-4d34-ab48-68f74a9d7c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:41.369056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06ff37-b937-4d34-ab48-68f74a9d7c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:41.369891 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06ff37-b937-4d34-ab48-68f74a9d7c26 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=380d6fe7057d44bfb20b0ae2bb1e664f) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:41.370062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 415/837] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:41.370199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06ff37-b937-4d34-ab48-68f74a9d7c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:41.371637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06ff37-b937-4d34-ab48-68f74a9d7c26 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:41.380317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 423/838] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:41 2022] GET /identity/v3/role_assignments?user.id=380d6fe7057d44bfb20b0ae2bb1e664f&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:41.381888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1cef1-f678-4207-a74e-b1fe1dec66d6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:41.454344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:41.504401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1cef1-f678-4207-a74e-b1fe1dec66d6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:41.542833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5baf4-3737-402e-be54-79e7bde515e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:41.543667 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5baf4-3737-402e-be54-79e7bde515e9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:41.543950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5baf4-3737-402e-be54-79e7bde515e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:41.544230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5baf4-3737-402e-be54-79e7bde515e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:41.552812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b5baf4-3737-402e-be54-79e7bde515e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:41.554257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b5baf4-3737-402e-be54-79e7bde515e9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e6b5baf4-3737-402e-be54-79e7bde515e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:41.570002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:41.571049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:41.571996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 416/839] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jul 21 17:21:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:41.577497 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06ff37-b937-4d34-ab48-68f74a9d7c26 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:41.680887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06ff37-b937-4d34-ab48-68f74a9d7c26 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:41.682459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:41.717197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84802f47-9c19-4044-a30b-22856aae2a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:41.717876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802f47-9c19-4044-a30b-22856aae2a87 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:41.718157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802f47-9c19-4044-a30b-22856aae2a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:41.718430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802f47-9c19-4044-a30b-22856aae2a87 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:41.719271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802f47-9c19-4044-a30b-22856aae2a87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:41.719580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802f47-9c19-4044-a30b-22856aae2a87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:41.721034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802f47-9c19-4044-a30b-22856aae2a87 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:41.739446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 424/840] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jul 21 17:21:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:41.745599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5baf4-3737-402e-be54-79e7bde515e9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:41.854431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5baf4-3737-402e-be54-79e7bde515e9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:41.879403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:41.891571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a694e-62d9-4b5c-8053-8641434b138c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:41.892226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a694e-62d9-4b5c-8053-8641434b138c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:41.892539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a694e-62d9-4b5c-8053-8641434b138c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:41.892817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a694e-62d9-4b5c-8053-8641434b138c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:41.964439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a694e-62d9-4b5c-8053-8641434b138c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:41.965902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a694e-62d9-4b5c-8053-8641434b138c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:41.969193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced6eb-d471-440f-ab61-1ada2f717ff5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:41.969572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced6eb-d471-440f-ab61-1ada2f717ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:41.969913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced6eb-d471-440f-ab61-1ada2f717ff5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:41.971557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 425/841] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:41.976355 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef996efd-eb51-422a-a566-e1f39c45cc2a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:41.976645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef996efd-eb51-422a-a566-e1f39c45cc2a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:41.977457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef996efd-eb51-422a-a566-e1f39c45cc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:41.994228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ef996efd-eb51-422a-a566-e1f39c45cc2a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:41.995454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ef996efd-eb51-422a-a566-e1f39c45cc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:42.065605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 426/842] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:42.066826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780836e-c16c-479b-82c6-0100f724b8c9 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.067118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780836e-c16c-479b-82c6-0100f724b8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.067398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780836e-c16c-479b-82c6-0100f724b8c9 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.068840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 427/843] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:42.074901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84154342-c649-42e5-ab2a-7edf7a21311a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.075192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84154342-c649-42e5-ab2a-7edf7a21311a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.075470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84154342-c649-42e5-ab2a-7edf7a21311a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.092423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-84154342-c649-42e5-ab2a-7edf7a21311a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:42.093615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-84154342-c649-42e5-ab2a-7edf7a21311a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:42.097631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 417/844] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:21:42.098501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a694e-62d9-4b5c-8053-8641434b138c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:42.126465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:42.167531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 428/845] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:42.168287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84154342-c649-42e5-ab2a-7edf7a21311a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:42.191700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:42.218096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a694e-62d9-4b5c-8053-8641434b138c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:42.254392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2881d-3644-4f57-a5dd-d6a6e6468318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:42.255427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2881d-3644-4f57-a5dd-d6a6e6468318 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.255708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2881d-3644-4f57-a5dd-d6a6e6468318 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.255981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2881d-3644-4f57-a5dd-d6a6e6468318 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.256654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2881d-3644-4f57-a5dd-d6a6e6468318 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:42.258005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2881d-3644-4f57-a5dd-d6a6e6468318 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-b1e2881d-3644-4f57-a5dd-d6a6e6468318 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:42.270166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:42.271161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 17:21:42.271996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:42.275801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84154342-c649-42e5-ab2a-7edf7a21311a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:42.276772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 418/846] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:21:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:42.277819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2881d-3644-4f57-a5dd-d6a6e6468318 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:42.313564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb21f56-27c2-4134-b4db-a1d6ebf5d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:42.315484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb21f56-27c2-4134-b4db-a1d6ebf5d2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.315770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb21f56-27c2-4134-b4db-a1d6ebf5d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.316046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb21f56-27c2-4134-b4db-a1d6ebf5d2a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.327577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb21f56-27c2-4134-b4db-a1d6ebf5d2a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:42.329062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb21f56-27c2-4134-b4db-a1d6ebf5d2a2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-4eb21f56-27c2-4134-b4db-a1d6ebf5d2a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:42.347273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:42.348154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:42.349094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:42.349094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:42.349094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:42.349094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:42.349094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 429/847] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:42.349388 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb21f56-27c2-4134-b4db-a1d6ebf5d2a2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:42.373782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:42.387543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2881d-3644-4f57-a5dd-d6a6e6468318 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:42.424057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5bfed-1a82-4515-80ef-66c090609e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:42.424883 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5bfed-1a82-4515-80ef-66c090609e2e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.425167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5bfed-1a82-4515-80ef-66c090609e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.425448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5bfed-1a82-4515-80ef-66c090609e2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.428829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:42.437347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5bfed-1a82-4515-80ef-66c090609e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f7b5bfed-1a82-4515-80ef-66c090609e2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:42.444874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:42.452011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 419/848] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:42.452917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5bfed-1a82-4515-80ef-66c090609e2e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:42.466907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb21f56-27c2-4134-b4db-a1d6ebf5d2a2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:42.504176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf69bcc-172f-4b61-a95b-300aa89ef598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:42.504875 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf69bcc-172f-4b61-a95b-300aa89ef598 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.505566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf69bcc-172f-4b61-a95b-300aa89ef598 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.505847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf69bcc-172f-4b61-a95b-300aa89ef598 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.506554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf69bcc-172f-4b61-a95b-300aa89ef598 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:42.506860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf69bcc-172f-4b61-a95b-300aa89ef598 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:42.508365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf69bcc-172f-4b61-a95b-300aa89ef598 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:42.512970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 430/849] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:21:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:42.517945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf69bcc-172f-4b61-a95b-300aa89ef598 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:42.557047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5bfed-1a82-4515-80ef-66c090609e2e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:42.594364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297104bf-495c-4f38-8d44-022baeca14cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:42.595741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297104bf-495c-4f38-8d44-022baeca14cf None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.596169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297104bf-495c-4f38-8d44-022baeca14cf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.596889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297104bf-495c-4f38-8d44-022baeca14cf None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.598052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297104bf-495c-4f38-8d44-022baeca14cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:42.598504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297104bf-495c-4f38-8d44-022baeca14cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:42.600556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297104bf-495c-4f38-8d44-022baeca14cf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:42.617955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 420/850] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:42.619060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf69bcc-172f-4b61-a95b-300aa89ef598 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:42.619191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297104bf-495c-4f38-8d44-022baeca14cf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:42.636235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:42.656540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba8634-0363-4212-a3ca-675568004c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:42.657218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba8634-0363-4212-a3ca-675568004c0b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.657506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba8634-0363-4212-a3ca-675568004c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.657785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba8634-0363-4212-a3ca-675568004c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.658539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba8634-0363-4212-a3ca-675568004c0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:42.658853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba8634-0363-4212-a3ca-675568004c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:42.660228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba8634-0363-4212-a3ca-675568004c0b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:42.662490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:42.673432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 431/851] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:42.674321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba8634-0363-4212-a3ca-675568004c0b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:42.739509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297104bf-495c-4f38-8d44-022baeca14cf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:42.776440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a82c8-adb3-46c3-a10e-2b308840b018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:42.777272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a82c8-adb3-46c3-a10e-2b308840b018 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.777564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a82c8-adb3-46c3-a10e-2b308840b018 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.777844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a82c8-adb3-46c3-a10e-2b308840b018 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.778556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a82c8-adb3-46c3-a10e-2b308840b018 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:42.778754 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba8634-0363-4212-a3ca-675568004c0b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:42.778855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a82c8-adb3-46c3-a10e-2b308840b018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:42.780255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a82c8-adb3-46c3-a10e-2b308840b018 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:42.784888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 421/852] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:21:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:42.785660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a82c8-adb3-46c3-a10e-2b308840b018 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:42.832222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae4786-df08-45b3-9967-364abcc17469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:42.833583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae4786-df08-45b3-9967-364abcc17469 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.834912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae4786-df08-45b3-9967-364abcc17469 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.835348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae4786-df08-45b3-9967-364abcc17469 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.845524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae4786-df08-45b3-9967-364abcc17469 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:42.847648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae4786-df08-45b3-9967-364abcc17469 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b7ae4786-df08-45b3-9967-364abcc17469 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:42.870656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:42.871817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:42.872984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:42.872984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:42.872984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:42.872984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:42.872984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:42.872984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:42.872984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:42.872984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:42.876249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 432/853] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:42.882493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae4786-df08-45b3-9967-364abcc17469 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:42.885372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:42.931862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a82c8-adb3-46c3-a10e-2b308840b018 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:42.974298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22500958-d37d-4b73-b4d1-e2e36c043790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:42.975282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22500958-d37d-4b73-b4d1-e2e36c043790 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:42.975550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22500958-d37d-4b73-b4d1-e2e36c043790 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:42.975656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:42.975819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22500958-d37d-4b73-b4d1-e2e36c043790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:42.991503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22500958-d37d-4b73-b4d1-e2e36c043790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:42.993168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22500958-d37d-4b73-b4d1-e2e36c043790 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-22500958-d37d-4b73-b4d1-e2e36c043790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:43.011423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:43.012441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:43.013355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:43.013355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:43.013355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:43.013355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:43.013355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 422/854] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:43.018313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22500958-d37d-4b73-b4d1-e2e36c043790 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:43.041942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae4786-df08-45b3-9967-364abcc17469 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:43.088317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa41ba-b5e6-4d98-9dab-cacf902df106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:43.089844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa41ba-b5e6-4d98-9dab-cacf902df106 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:43.090693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa41ba-b5e6-4d98-9dab-cacf902df106 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:43.091195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa41ba-b5e6-4d98-9dab-cacf902df106 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:43.092381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa41ba-b5e6-4d98-9dab-cacf902df106 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:43.093630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa41ba-b5e6-4d98-9dab-cacf902df106 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:43.095265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa41ba-b5e6-4d98-9dab-cacf902df106 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:43.116741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 433/855] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:43.117748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa41ba-b5e6-4d98-9dab-cacf902df106 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:43.130489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22500958-d37d-4b73-b4d1-e2e36c043790 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:43.155767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:43.169516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0c196-592e-4f50-b589-225da2d48e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:43.170241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0c196-592e-4f50-b589-225da2d48e47 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:43.170521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0c196-592e-4f50-b589-225da2d48e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:43.170798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0c196-592e-4f50-b589-225da2d48e47 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:43.171573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0c196-592e-4f50-b589-225da2d48e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:43.171882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0c196-592e-4f50-b589-225da2d48e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:43.173647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0c196-592e-4f50-b589-225da2d48e47 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:43.191391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 423/856] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:43.192449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0c196-592e-4f50-b589-225da2d48e47 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:43.261730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa41ba-b5e6-4d98-9dab-cacf902df106 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:43.285382 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:43.307419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb429dd9-0d08-414e-99d7-0f6323dd22a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:43.307843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0c196-592e-4f50-b589-225da2d48e47 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:43.308121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429dd9-0d08-414e-99d7-0f6323dd22a7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:43.308448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429dd9-0d08-414e-99d7-0f6323dd22a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:43.308723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429dd9-0d08-414e-99d7-0f6323dd22a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:43.309552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb429dd9-0d08-414e-99d7-0f6323dd22a7 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=75f11eaa4b504d0aa4018fb4ffbbd3a2) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:43.309860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb429dd9-0d08-414e-99d7-0f6323dd22a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:43.311280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb429dd9-0d08-414e-99d7-0f6323dd22a7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:43.324329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 434/857] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:43 2022] GET /identity/v3/role_assignments?user.id=75f11eaa4b504d0aa4018fb4ffbbd3a2&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:43.356458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cb36b-a8f2-4332-9f3d-813862052717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:43.357326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb36b-a8f2-4332-9f3d-813862052717 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:43.357608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb36b-a8f2-4332-9f3d-813862052717 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:43.357882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb36b-a8f2-4332-9f3d-813862052717 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:43.366673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cb36b-a8f2-4332-9f3d-813862052717 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:43.368382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cb36b-a8f2-4332-9f3d-813862052717 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-466cb36b-a8f2-4332-9f3d-813862052717 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:43.384717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:43.385782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:43.386762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 424/858] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:21:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:43.393816 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb429dd9-0d08-414e-99d7-0f6323dd22a7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:43.514444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb429dd9-0d08-414e-99d7-0f6323dd22a7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:43.551081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38957792-a510-4560-9422-1cb85bbcd1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:43.552160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38957792-a510-4560-9422-1cb85bbcd1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:43.552679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38957792-a510-4560-9422-1cb85bbcd1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:43.553156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38957792-a510-4560-9422-1cb85bbcd1c9 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:43.554200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38957792-a510-4560-9422-1cb85bbcd1c9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:43.554733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38957792-a510-4560-9422-1cb85bbcd1c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:43.556625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38957792-a510-4560-9422-1cb85bbcd1c9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:43.565552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703c1ad-e8ca-4261-8ba5-e42f35008dbe None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:43.565839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703c1ad-e8ca-4261-8ba5-e42f35008dbe None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:43.566105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703c1ad-e8ca-4261-8ba5-e42f35008dbe None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:43.567609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 425/859] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:43.573174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca275e1-cce7-45aa-af61-804a93fbf0c7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:43.573457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca275e1-cce7-45aa-af61-804a93fbf0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:43.573989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca275e1-cce7-45aa-af61-804a93fbf0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:43.581650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 435/860] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Jul 21 17:21:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:43.589380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38957792-a510-4560-9422-1cb85bbcd1c9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:43.597928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6ca275e1-cce7-45aa-af61-804a93fbf0c7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:43.599136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6ca275e1-cce7-45aa-af61-804a93fbf0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:43.642457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:43.645380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:43.681348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 426/861] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:43.704130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca275e1-cce7-45aa-af61-804a93fbf0c7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:43.717739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38957792-a510-4560-9422-1cb85bbcd1c9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:43.754350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363db586-06ef-4469-83a9-19eabf106e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:43.754964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363db586-06ef-4469-83a9-19eabf106e39 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:43.755252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363db586-06ef-4469-83a9-19eabf106e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:43.755533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363db586-06ef-4469-83a9-19eabf106e39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:43.811400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca275e1-cce7-45aa-af61-804a93fbf0c7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:43.829638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363db586-06ef-4469-83a9-19eabf106e39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:43.831111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363db586-06ef-4469-83a9-19eabf106e39 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:43.849258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd3045-6d1f-44a4-9dee-1f365abc35d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:43.850517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd3045-6d1f-44a4-9dee-1f365abc35d5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:43.850999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd3045-6d1f-44a4-9dee-1f365abc35d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:43.851471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd3045-6d1f-44a4-9dee-1f365abc35d5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:43.852405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd3045-6d1f-44a4-9dee-1f365abc35d5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:43.854142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd3045-6d1f-44a4-9dee-1f365abc35d5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e1bd3045-6d1f-44a4-9dee-1f365abc35d5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:43.866979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:43.868678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 17:21:43.869525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:43.876858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:43.877268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 427/862] 10.222.0.57 () {56 vars in 1132 bytes} [Thu Jul 21 17:21:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:43.878398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd3045-6d1f-44a4-9dee-1f365abc35d5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:43.921592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:43.958015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 436/863] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:21:43.959120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363db586-06ef-4469-83a9-19eabf106e39 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:43.997922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd3045-6d1f-44a4-9dee-1f365abc35d5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:44.034109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c2a2c-2962-4f14-89b4-debb34dae985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:44.035121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c2a2c-2962-4f14-89b4-debb34dae985 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:44.035605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c2a2c-2962-4f14-89b4-debb34dae985 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:44.036113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c2a2c-2962-4f14-89b4-debb34dae985 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:44.037110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c2a2c-2962-4f14-89b4-debb34dae985 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:44.037645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c2a2c-2962-4f14-89b4-debb34dae985 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:44.039199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c2a2c-2962-4f14-89b4-debb34dae985 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:44.049508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 428/864] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:21:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:44.050598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c2a2c-2962-4f14-89b4-debb34dae985 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:44.066555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363db586-06ef-4469-83a9-19eabf106e39 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:44.103721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03448f-9a41-4215-956e-0dbba1e1640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:44.104546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03448f-9a41-4215-956e-0dbba1e1640f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:44.104833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03448f-9a41-4215-956e-0dbba1e1640f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:44.105110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03448f-9a41-4215-956e-0dbba1e1640f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:44.114043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:44.118720 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03448f-9a41-4215-956e-0dbba1e1640f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:44.120256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03448f-9a41-4215-956e-0dbba1e1640f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-be03448f-9a41-4215-956e-0dbba1e1640f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:44.139292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:44.140255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:44.141184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:44.141184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:44.141184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:44.141184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:44.141184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 437/865] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:44.146823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03448f-9a41-4215-956e-0dbba1e1640f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:44.170744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c2a2c-2962-4f14-89b4-debb34dae985 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:44.172796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:44.210528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bce5e-b4f5-4cb4-b2ce-74d10d24ba75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:44.211612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bce5e-b4f5-4cb4-b2ce-74d10d24ba75 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:44.212176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bce5e-b4f5-4cb4-b2ce-74d10d24ba75 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:44.212679 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bce5e-b4f5-4cb4-b2ce-74d10d24ba75 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:44.213647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bce5e-b4f5-4cb4-b2ce-74d10d24ba75 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:44.214155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bce5e-b4f5-4cb4-b2ce-74d10d24ba75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:44.215922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bce5e-b4f5-4cb4-b2ce-74d10d24ba75 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:44.228807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 429/866] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:21:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:44.266965 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03448f-9a41-4215-956e-0dbba1e1640f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:44.308869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884c34d-6ee7-4968-b0a1-8d4b844f8252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:44.309607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884c34d-6ee7-4968-b0a1-8d4b844f8252 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:44.309894 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884c34d-6ee7-4968-b0a1-8d4b844f8252 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:44.310165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884c34d-6ee7-4968-b0a1-8d4b844f8252 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:44.311168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884c34d-6ee7-4968-b0a1-8d4b844f8252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:44.311480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884c34d-6ee7-4968-b0a1-8d4b844f8252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:44.312932 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884c34d-6ee7-4968-b0a1-8d4b844f8252 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:44.327413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 438/867] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:44.333192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bce5e-b4f5-4cb4-b2ce-74d10d24ba75 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:44.444080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bce5e-b4f5-4cb4-b2ce-74d10d24ba75 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:44.481195 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944464d-b337-4fec-95cc-c0220709f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:44.482355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944464d-b337-4fec-95cc-c0220709f8fc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:44.482858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944464d-b337-4fec-95cc-c0220709f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:44.483360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944464d-b337-4fec-95cc-c0220709f8fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:44.490620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944464d-b337-4fec-95cc-c0220709f8fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:44.492450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944464d-b337-4fec-95cc-c0220709f8fc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8944464d-b337-4fec-95cc-c0220709f8fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:44.507921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:44.508914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:44.509792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:44.509792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:44.509792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:44.509792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:44.509792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:44.509792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:44.509792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:44.509792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:44.512314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 430/868] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:44.520381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884c34d-6ee7-4968-b0a1-8d4b844f8252 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:44.580404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:44.634108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884c34d-6ee7-4968-b0a1-8d4b844f8252 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:44.671372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd1afe-da24-444e-9cdd-77657b45724d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:44.672046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd1afe-da24-444e-9cdd-77657b45724d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:44.672354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd1afe-da24-444e-9cdd-77657b45724d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:44.672667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd1afe-da24-444e-9cdd-77657b45724d None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:44.673422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd1afe-da24-444e-9cdd-77657b45724d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:44.673750 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd1afe-da24-444e-9cdd-77657b45724d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:44.675177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd1afe-da24-444e-9cdd-77657b45724d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:44.688527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 439/869] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:44.694687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944464d-b337-4fec-95cc-c0220709f8fc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:44.724628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:44.806323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944464d-b337-4fec-95cc-c0220709f8fc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:44.841858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37489783-8a2e-4227-87a4-a45d92dbef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:44.842438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37489783-8a2e-4227-87a4-a45d92dbef3f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:44.842717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37489783-8a2e-4227-87a4-a45d92dbef3f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:44.842991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37489783-8a2e-4227-87a4-a45d92dbef3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:44.843824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37489783-8a2e-4227-87a4-a45d92dbef3f None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=8a26412cf9f64b1bbbd8e595b55bcebc) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:44.844131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37489783-8a2e-4227-87a4-a45d92dbef3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:44.845571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37489783-8a2e-4227-87a4-a45d92dbef3f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:44.852647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 431/870] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:44 2022] GET /identity/v3/role_assignments?user.id=8a26412cf9f64b1bbbd8e595b55bcebc&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:44.999247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f383f96-38df-4448-87d4-414a857001c5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:44.999573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f383f96-38df-4448-87d4-414a857001c5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:45.000283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f383f96-38df-4448-87d4-414a857001c5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:45.003276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 440/871] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:45.012021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98187c81-2870-4dfa-8847-2095a55fa8d8 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:45.012589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98187c81-2870-4dfa-8847-2095a55fa8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:45.013008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98187c81-2870-4dfa-8847-2095a55fa8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:45.037740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-98187c81-2870-4dfa-8847-2095a55fa8d8 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:45.037972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-98187c81-2870-4dfa-8847-2095a55fa8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:45.111120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 432/872] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:45.134588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f383f96-38df-4448-87d4-414a857001c5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:45.239207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f383f96-38df-4448-87d4-414a857001c5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:45.253549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:45.277230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd02ce-af38-401f-8e59-5cca565d8e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:45.277880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd02ce-af38-401f-8e59-5cca565d8e17 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:45.278171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd02ce-af38-401f-8e59-5cca565d8e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:45.278452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd02ce-af38-401f-8e59-5cca565d8e17 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:45.279188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd02ce-af38-401f-8e59-5cca565d8e17 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:45.280597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd02ce-af38-401f-8e59-5cca565d8e17 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:45.304443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 441/873] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:21:45 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:45.540872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc80b3-c391-40ce-a346-dc38115c483e None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:45.541585 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc80b3-c391-40ce-a346-dc38115c483e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:45.542082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc80b3-c391-40ce-a346-dc38115c483e None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:45.543909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 433/874] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:45.549263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1dd8f-2305-4c27-8617-345f98cf6af1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:45.549811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1dd8f-2305-4c27-8617-345f98cf6af1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:45.550299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1dd8f-2305-4c27-8617-345f98cf6af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:45.567996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2da1dd8f-2305-4c27-8617-345f98cf6af1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:45.569522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2da1dd8f-2305-4c27-8617-345f98cf6af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:45.652220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 442/875] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:45.675961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc80b3-c391-40ce-a346-dc38115c483e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:45.699754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:45.784324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc80b3-c391-40ce-a346-dc38115c483e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:45.821875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9341b9-269e-4165-9b97-8d2674bb2816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:45.822651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9341b9-269e-4165-9b97-8d2674bb2816 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:45.822932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9341b9-269e-4165-9b97-8d2674bb2816 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:45.823206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9341b9-269e-4165-9b97-8d2674bb2816 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:45.834124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9341b9-269e-4165-9b97-8d2674bb2816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-ec9341b9-269e-4165-9b97-8d2674bb2816 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:45.842600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:45.843566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:45.844458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 434/876] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jul 21 17:21:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:45.850448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1dd8f-2305-4c27-8617-345f98cf6af1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:45.960671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1dd8f-2305-4c27-8617-345f98cf6af1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:45.968648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:45.999585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5454797-b49c-4087-b270-2606d6654628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:46.000969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5454797-b49c-4087-b270-2606d6654628 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:46.001285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5454797-b49c-4087-b270-2606d6654628 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:46.001562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5454797-b49c-4087-b270-2606d6654628 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:46.002834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5454797-b49c-4087-b270-2606d6654628 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:46.003432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5454797-b49c-4087-b270-2606d6654628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:46.005985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5454797-b49c-4087-b270-2606d6654628 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:46.018801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 443/877] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jul 21 17:21:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:46.024617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9341b9-269e-4165-9b97-8d2674bb2816 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:46.114508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:46.135359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9341b9-269e-4165-9b97-8d2674bb2816 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:46.171997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c46f0e-47b1-4df6-9752-a40f4d77b761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:46.179724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c46f0e-47b1-4df6-9752-a40f4d77b761 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:46.180059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c46f0e-47b1-4df6-9752-a40f4d77b761 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:46.180366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c46f0e-47b1-4df6-9752-a40f4d77b761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:46.191750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c46f0e-47b1-4df6-9752-a40f4d77b761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:46.193215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c46f0e-47b1-4df6-9752-a40f4d77b761 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-11c46f0e-47b1-4df6-9752-a40f4d77b761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:46.210834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:46.212275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:46.213218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:46.213218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:46.213218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:46.213218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:46.213218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 435/878] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:46.218546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5454797-b49c-4087-b270-2606d6654628 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:46.320207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5454797-b49c-4087-b270-2606d6654628 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:46.355291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d0e3d-e09c-447d-838e-dcac539c8d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:46.355952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d0e3d-e09c-447d-838e-dcac539c8d18 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:46.356277 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d0e3d-e09c-447d-838e-dcac539c8d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:46.356573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d0e3d-e09c-447d-838e-dcac539c8d18 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:46.357290 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d0e3d-e09c-447d-838e-dcac539c8d18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:46.357592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d0e3d-e09c-447d-838e-dcac539c8d18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:46.359003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d0e3d-e09c-447d-838e-dcac539c8d18 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:46.371311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 444/879] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:46.376740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c46f0e-47b1-4df6-9752-a40f4d77b761 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:46.393618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901299-36d3-414d-80b2-42ac93c6eafe None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:46.393912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901299-36d3-414d-80b2-42ac93c6eafe None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:46.394161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901299-36d3-414d-80b2-42ac93c6eafe None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:46.395755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 445/880] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:46.401089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7c633-7f6a-457c-adaf-9156b56b34e1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:46.401372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7c633-7f6a-457c-adaf-9156b56b34e1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:46.401650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7c633-7f6a-457c-adaf-9156b56b34e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:46.419522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1de7c633-7f6a-457c-adaf-9156b56b34e1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:46.420824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1de7c633-7f6a-457c-adaf-9156b56b34e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:46.440105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:46.487969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c46f0e-47b1-4df6-9752-a40f4d77b761 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:46.519487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 446/881] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:46 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:46.525608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:46.531443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3e19f-bdf4-4dc9-a822-324dfeb2379f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:46.532544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3e19f-bdf4-4dc9-a822-324dfeb2379f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:46.532988 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3e19f-bdf4-4dc9-a822-324dfeb2379f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:46.533321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3e19f-bdf4-4dc9-a822-324dfeb2379f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:46.542729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3e19f-bdf4-4dc9-a822-324dfeb2379f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:46.544211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3e19f-bdf4-4dc9-a822-324dfeb2379f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:46.546167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7c633-7f6a-457c-adaf-9156b56b34e1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e7b3e19f-bdf4-4dc9-a822-324dfeb2379f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:46.564630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:46.565640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:46.566591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:46.566591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:46.566591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:46.566591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:46.566591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:46.566591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:46.566591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:46.566591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:46.566591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:46.566591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:46.569339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 436/882] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jul 21 17:21:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:46.577414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3e19f-bdf4-4dc9-a822-324dfeb2379f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:46.670875 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7c633-7f6a-457c-adaf-9156b56b34e1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:46.699184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3e19f-bdf4-4dc9-a822-324dfeb2379f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:46.713970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f112658-39a7-42de-88dc-fb4d418d3c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:46.715018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f112658-39a7-42de-88dc-fb4d418d3c13 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:46.715018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f112658-39a7-42de-88dc-fb4d418d3c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:46.715396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f112658-39a7-42de-88dc-fb4d418d3c13 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:46.716256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f112658-39a7-42de-88dc-fb4d418d3c13 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:46.717744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f112658-39a7-42de-88dc-fb4d418d3c13 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:46.740401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 447/883] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:21:46 2022] POST /identity/v3/services => generated 269 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:21:46.754500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298407d6-12f4-425b-b3fc-6028758619d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:46.755238 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298407d6-12f4-425b-b3fc-6028758619d4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:46.755520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298407d6-12f4-425b-b3fc-6028758619d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:46.755795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298407d6-12f4-425b-b3fc-6028758619d4 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:46.756720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298407d6-12f4-425b-b3fc-6028758619d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:46.757034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298407d6-12f4-425b-b3fc-6028758619d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:46.759171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298407d6-12f4-425b-b3fc-6028758619d4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:46.780451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 437/884] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jul 21 17:21:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:46.786733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f112658-39a7-42de-88dc-fb4d418d3c13 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:46.808700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:46.906049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f112658-39a7-42de-88dc-fb4d418d3c13 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:46.944927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d898ab5-75b4-4d0a-a625-87025b3afc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:46.946626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d898ab5-75b4-4d0a-a625-87025b3afc42 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:46.946919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d898ab5-75b4-4d0a-a625-87025b3afc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:46.947197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d898ab5-75b4-4d0a-a625-87025b3afc42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.021090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d898ab5-75b4-4d0a-a625-87025b3afc42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:47.022997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d898ab5-75b4-4d0a-a625-87025b3afc42 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:47.035531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:47.057060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068d535-6667-4ce1-9502-4605fccfc7cc None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.057570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068d535-6667-4ce1-9502-4605fccfc7cc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.057646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068d535-6667-4ce1-9502-4605fccfc7cc None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.059239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 438/885] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:47.064336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817f7d0-9899-4d2e-b11c-18f4ffcb260b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.064633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817f7d0-9899-4d2e-b11c-18f4ffcb260b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.064913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817f7d0-9899-4d2e-b11c-18f4ffcb260b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.082441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d817f7d0-9899-4d2e-b11c-18f4ffcb260b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:47.083732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d817f7d0-9899-4d2e-b11c-18f4ffcb260b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:47.140152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 448/886] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:46 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:21:47.140879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:47.146351 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d898ab5-75b4-4d0a-a625-87025b3afc42 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:47.170844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 439/887] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:47.197891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817f7d0-9899-4d2e-b11c-18f4ffcb260b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:47.280630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d898ab5-75b4-4d0a-a625-87025b3afc42 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:47.292853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:47.316415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817f7d0-9899-4d2e-b11c-18f4ffcb260b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:47.324952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac192f1-c374-41b8-b4b2-a225c5f0d7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:47.326443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac192f1-c374-41b8-b4b2-a225c5f0d7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.326937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac192f1-c374-41b8-b4b2-a225c5f0d7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.327514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac192f1-c374-41b8-b4b2-a225c5f0d7c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.340116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac192f1-c374-41b8-b4b2-a225c5f0d7c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:47.342195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac192f1-c374-41b8-b4b2-a225c5f0d7c5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:47.356870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a4299-5f69-44d8-9caa-63006492a003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:47.357686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a4299-5f69-44d8-9caa-63006492a003 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.357970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a4299-5f69-44d8-9caa-63006492a003 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.358249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a4299-5f69-44d8-9caa-63006492a003 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.358912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a4299-5f69-44d8-9caa-63006492a003 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:47.360283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a4299-5f69-44d8-9caa-63006492a003 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6ac192f1-c374-41b8-b4b2-a225c5f0d7c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:47.364179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:47.365183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:47.366102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:47.366102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:47.366102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:47.366102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a83a4299-5f69-44d8-9caa-63006492a003 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:47.372639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:47.373620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:47.374448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 440/889] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:21:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:47.375266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a4299-5f69-44d8-9caa-63006492a003 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:47.375881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 449/889] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:47.382003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac192f1-c374-41b8-b4b2-a225c5f0d7c5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:47.425596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:47.491490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a4299-5f69-44d8-9caa-63006492a003 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:47.509253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac192f1-c374-41b8-b4b2-a225c5f0d7c5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:47.528606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f9902-bb4c-45ea-b965-f4cae57113a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:47.529570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f9902-bb4c-45ea-b965-f4cae57113a3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.530030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f9902-bb4c-45ea-b965-f4cae57113a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.530474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f9902-bb4c-45ea-b965-f4cae57113a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.531381 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f9902-bb4c-45ea-b965-f4cae57113a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:47.531848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f9902-bb4c-45ea-b965-f4cae57113a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:47.533445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f9902-bb4c-45ea-b965-f4cae57113a3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:47.547686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 441/890] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:47.549399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11848d4a-8d5d-4a9b-b353-231ef5bab4fb None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.549819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11848d4a-8d5d-4a9b-b353-231ef5bab4fb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.550222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11848d4a-8d5d-4a9b-b353-231ef5bab4fb None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.551990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 442/891] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:47.553055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11848d4a-8d5d-4a9b-b353-231ef5bab4fb None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:47.661947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc4f41-86c8-41ee-97c6-59494cb3795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:47.662602 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc4f41-86c8-41ee-97c6-59494cb3795d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.662881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc4f41-86c8-41ee-97c6-59494cb3795d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.663155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc4f41-86c8-41ee-97c6-59494cb3795d None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.663843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc4f41-86c8-41ee-97c6-59494cb3795d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:47.664170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc4f41-86c8-41ee-97c6-59494cb3795d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:47.665876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc4f41-86c8-41ee-97c6-59494cb3795d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:47.666323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:47.674998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 450/892] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jul 21 17:21:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:47.674998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c5c35-923d-4de5-83f7-5310075f7c2a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.674998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c5c35-923d-4de5-83f7-5310075f7c2a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.674998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c5c35-923d-4de5-83f7-5310075f7c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.676044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11848d4a-8d5d-4a9b-b353-231ef5bab4fb None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:47.691430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-461c5c35-923d-4de5-83f7-5310075f7c2a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:47.692651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-461c5c35-923d-4de5-83f7-5310075f7c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:47.714488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:47.715414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcdd36-3583-42f2-957f-b4087dd8cb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:47.716633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcdd36-3583-42f2-957f-b4087dd8cb88 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.717042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcdd36-3583-42f2-957f-b4087dd8cb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.717322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcdd36-3583-42f2-957f-b4087dd8cb88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.724213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcdd36-3583-42f2-957f-b4087dd8cb88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:47.725667 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcdd36-3583-42f2-957f-b4087dd8cb88 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-77dcdd36-3583-42f2-957f-b4087dd8cb88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:47.740056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:47.740990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:47.741958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:47.741958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:47.741958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:47.741958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:47.741958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:47.741958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:47.741958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:47.741958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:47.741958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 443/893] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:47.742955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcdd36-3583-42f2-957f-b4087dd8cb88 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:47.771230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 451/894] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:47.771908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c5c35-923d-4de5-83f7-5310075f7c2a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:47.865817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcdd36-3583-42f2-957f-b4087dd8cb88 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:47.872380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c5c35-923d-4de5-83f7-5310075f7c2a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:47.902692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2aadac-409e-4904-9f66-7ddc83453808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:47.903336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2aadac-409e-4904-9f66-7ddc83453808 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.903645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2aadac-409e-4904-9f66-7ddc83453808 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.903886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2aadac-409e-4904-9f66-7ddc83453808 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.904626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2aadac-409e-4904-9f66-7ddc83453808 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:47.904987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2aadac-409e-4904-9f66-7ddc83453808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:47.905582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5c79a-6023-4216-b3f6-972d9211e768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:47.906233 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5c79a-6023-4216-b3f6-972d9211e768 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:47.906344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2aadac-409e-4904-9f66-7ddc83453808 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:47.906514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5c79a-6023-4216-b3f6-972d9211e768 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:47.906786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5c79a-6023-4216-b3f6-972d9211e768 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:47.907564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5c79a-6023-4216-b3f6-972d9211e768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:47.907869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5c79a-6023-4216-b3f6-972d9211e768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:47.909303 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5c79a-6023-4216-b3f6-972d9211e768 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:47.917309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 444/895] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jul 21 17:21:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:47.917309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2aadac-409e-4904-9f66-7ddc83453808 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:47.938914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 452/896] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:47.940069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5c79a-6023-4216-b3f6-972d9211e768 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:47.953748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:47.963525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:48.055749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2aadac-409e-4904-9f66-7ddc83453808 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:48.078489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5c79a-6023-4216-b3f6-972d9211e768 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:48.096210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c8bad-1654-45d0-9624-0104312fd7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:48.097035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c8bad-1654-45d0-9624-0104312fd7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:48.097317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c8bad-1654-45d0-9624-0104312fd7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:48.097590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c8bad-1654-45d0-9624-0104312fd7e8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:48.098244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c8bad-1654-45d0-9624-0104312fd7e8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:48.099729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c8bad-1654-45d0-9624-0104312fd7e8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-ce4c8bad-1654-45d0-9624-0104312fd7e8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:48.113051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:48.114078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:48.114927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 445/897] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:21:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:48.120194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c8bad-1654-45d0-9624-0104312fd7e8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:48.120755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68697a5-4aab-48a5-8f2c-dac56b62081a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:48.121416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68697a5-4aab-48a5-8f2c-dac56b62081a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:48.121693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68697a5-4aab-48a5-8f2c-dac56b62081a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:48.121965 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68697a5-4aab-48a5-8f2c-dac56b62081a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:48.122798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68697a5-4aab-48a5-8f2c-dac56b62081a None admin] RBAC: Adding query filter params (role.id=085ea69bad9746939d7f6aad1904b3f3, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=75f11eaa4b504d0aa4018fb4ffbbd3a2) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:48.123107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68697a5-4aab-48a5-8f2c-dac56b62081a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:48.125128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68697a5-4aab-48a5-8f2c-dac56b62081a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:48.130518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 453/898] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:47 2022] GET /identity/v3/role_assignments?user.id=75f11eaa4b504d0aa4018fb4ffbbd3a2&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=085ea69bad9746939d7f6aad1904b3f3 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:48.231363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:48.251478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c8bad-1654-45d0-9624-0104312fd7e8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:48.289916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadf99b-c92a-43f8-8428-1c0a99ae8ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:48.290589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadf99b-c92a-43f8-8428-1c0a99ae8ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:48.290878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadf99b-c92a-43f8-8428-1c0a99ae8ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:48.291157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadf99b-c92a-43f8-8428-1c0a99ae8ec5 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:48.291868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadf99b-c92a-43f8-8428-1c0a99ae8ec5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:48.292175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadf99b-c92a-43f8-8428-1c0a99ae8ec5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:48.293640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadf99b-c92a-43f8-8428-1c0a99ae8ec5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:48.299146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 446/899] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:21:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:48.304572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68697a5-4aab-48a5-8f2c-dac56b62081a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:48.392827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:48.413320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68697a5-4aab-48a5-8f2c-dac56b62081a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:48.449909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ce10c-d0e8-4652-afe0-95845bc535d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:48.450586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ce10c-d0e8-4652-afe0-95845bc535d6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:48.450873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ce10c-d0e8-4652-afe0-95845bc535d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:48.451150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ce10c-d0e8-4652-afe0-95845bc535d6 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:48.451861 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ce10c-d0e8-4652-afe0-95845bc535d6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:48.452171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ce10c-d0e8-4652-afe0-95845bc535d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:48.453580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ce10c-d0e8-4652-afe0-95845bc535d6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:48.460634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 454/900] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:21:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:48.469385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadf99b-c92a-43f8-8428-1c0a99ae8ec5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:48.576066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadf99b-c92a-43f8-8428-1c0a99ae8ec5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:48.610838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575570e1-dc37-4705-972e-c7e0701e44c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:48.611514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575570e1-dc37-4705-972e-c7e0701e44c0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:48.611794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575570e1-dc37-4705-972e-c7e0701e44c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:48.612069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575570e1-dc37-4705-972e-c7e0701e44c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:48.612903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575570e1-dc37-4705-972e-c7e0701e44c0 None admin] RBAC: Adding query filter params (interface=public, service_id=be1bca6338eb46f98ca375cff92b7f51, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:48.613220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575570e1-dc37-4705-972e-c7e0701e44c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:48.614588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575570e1-dc37-4705-972e-c7e0701e44c0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:48.620984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 447/901] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jul 21 17:21:48 2022] GET /identity/v3/endpoints?service_id=be1bca6338eb46f98ca375cff92b7f51&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:48.626692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ce10c-d0e8-4652-afe0-95845bc535d6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:48.729976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ce10c-d0e8-4652-afe0-95845bc535d6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:48.764379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5cf85-0d1e-4ab8-a51b-bda39abb5f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:48.765373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5cf85-0d1e-4ab8-a51b-bda39abb5f8e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:48.765871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5cf85-0d1e-4ab8-a51b-bda39abb5f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:48.766394 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5cf85-0d1e-4ab8-a51b-bda39abb5f8e None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:48.767300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5cf85-0d1e-4ab8-a51b-bda39abb5f8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:48.768890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5cf85-0d1e-4ab8-a51b-bda39abb5f8e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:48.775268 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 455/902] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jul 21 17:21:48 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:21:48.974481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a660d6b-1f5f-4898-93ad-ee4900f7449f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:48.974823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a660d6b-1f5f-4898-93ad-ee4900f7449f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:48.975040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a660d6b-1f5f-4898-93ad-ee4900f7449f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:48.977290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 448/903] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:48.982911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3b03a-788f-4d9d-a743-4a19c0b8093b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:48.983136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3b03a-788f-4d9d-a743-4a19c0b8093b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:48.983415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3b03a-788f-4d9d-a743-4a19c0b8093b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:49.001136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7ec3b03a-788f-4d9d-a743-4a19c0b8093b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:49.002318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7ec3b03a-788f-4d9d-a743-4a19c0b8093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:49.004307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:49.084408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 456/904] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:49.104183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a660d6b-1f5f-4898-93ad-ee4900f7449f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:49.130050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:49.220034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a660d6b-1f5f-4898-93ad-ee4900f7449f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:49.255268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf4219-6059-458e-ac7f-28cfc1625877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:49.256329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf4219-6059-458e-ac7f-28cfc1625877 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:49.256758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf4219-6059-458e-ac7f-28cfc1625877 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:49.257198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf4219-6059-458e-ac7f-28cfc1625877 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:49.258075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf4219-6059-458e-ac7f-28cfc1625877 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:49.259581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf4219-6059-458e-ac7f-28cfc1625877 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a2bf4219-6059-458e-ac7f-28cfc1625877 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:49.272241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:49.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 17:21:49.275037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:49.277446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 449/905] 10.222.0.57 () {56 vars in 1132 bytes} [Thu Jul 21 17:21:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:49.284324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3b03a-788f-4d9d-a743-4a19c0b8093b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:49.397533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3b03a-788f-4d9d-a743-4a19c0b8093b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:49.433921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff8c2a-a924-4584-8c70-bc8ad1c8b336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:49.434591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff8c2a-a924-4584-8c70-bc8ad1c8b336 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:49.434876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff8c2a-a924-4584-8c70-bc8ad1c8b336 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:49.435153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff8c2a-a924-4584-8c70-bc8ad1c8b336 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:49.435859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff8c2a-a924-4584-8c70-bc8ad1c8b336 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:49.436165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff8c2a-a924-4584-8c70-bc8ad1c8b336 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:49.437642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff8c2a-a924-4584-8c70-bc8ad1c8b336 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:49.448015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 457/906] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:21:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:49.449619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf4219-6059-458e-ac7f-28cfc1625877 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:49.532126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:49.551396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf4219-6059-458e-ac7f-28cfc1625877 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:49.585218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fce07-5082-4339-b3b0-33de83872a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:49.585967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fce07-5082-4339-b3b0-33de83872a2b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:49.586343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fce07-5082-4339-b3b0-33de83872a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:49.586623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fce07-5082-4339-b3b0-33de83872a2b None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:49.587334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fce07-5082-4339-b3b0-33de83872a2b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:49.587641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fce07-5082-4339-b3b0-33de83872a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:49.589030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fce07-5082-4339-b3b0-33de83872a2b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:49.598833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 450/907] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:21:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:49.600167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff8c2a-a924-4584-8c70-bc8ad1c8b336 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:49.603586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:49.704846 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff8c2a-a924-4584-8c70-bc8ad1c8b336 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:49.740361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebd48b-d4aa-4ed7-8025-9e8591df805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:49.741028 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebd48b-d4aa-4ed7-8025-9e8591df805f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:49.741313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebd48b-d4aa-4ed7-8025-9e8591df805f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:49.741606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebd48b-d4aa-4ed7-8025-9e8591df805f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:49.742451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebd48b-d4aa-4ed7-8025-9e8591df805f None admin] RBAC: Adding query filter params (interface=public, service_id=e0e497b4386c4704b10c42fb3c5abf30, region_id=RegionOne) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:49.742764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebd48b-d4aa-4ed7-8025-9e8591df805f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:49.744143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebd48b-d4aa-4ed7-8025-9e8591df805f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:49.753921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 458/908] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jul 21 17:21:49 2022] GET /identity/v3/endpoints?service_id=e0e497b4386c4704b10c42fb3c5abf30&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:49.755316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fce07-5082-4339-b3b0-33de83872a2b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:49.857732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fce07-5082-4339-b3b0-33de83872a2b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:49.897266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d95a8b-bf1c-4c81-aebc-aa40e72d2732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:49.898810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d95a8b-bf1c-4c81-aebc-aa40e72d2732 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:49.899320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d95a8b-bf1c-4c81-aebc-aa40e72d2732 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:49.899931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d95a8b-bf1c-4c81-aebc-aa40e72d2732 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:49.900937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d95a8b-bf1c-4c81-aebc-aa40e72d2732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:49.902505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d95a8b-bf1c-4c81-aebc-aa40e72d2732 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:49.909200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 451/909] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jul 21 17:21:49 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:21:50.124713 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a1c59-8aa5-422d-8b8c-8a58b27fd167 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:50.125070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a1c59-8aa5-422d-8b8c-8a58b27fd167 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:50.125270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a1c59-8aa5-422d-8b8c-8a58b27fd167 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:50.126885 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 459/910] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:50.131957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca22bef-921d-4231-bdb0-2ffa3fad303e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:50.132405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca22bef-921d-4231-bdb0-2ffa3fad303e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:50.132585 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca22bef-921d-4231-bdb0-2ffa3fad303e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:50.150658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3ca22bef-921d-4231-bdb0-2ffa3fad303e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:50.151835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3ca22bef-921d-4231-bdb0-2ffa3fad303e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:50.154056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:50.225334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 452/911] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:50.247907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a1c59-8aa5-422d-8b8c-8a58b27fd167 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:50.319559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:50.344789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6abfb-5b7a-47d5-9ca7-96f9d5ec798d None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:50.345424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6abfb-5b7a-47d5-9ca7-96f9d5ec798d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:50.345927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6abfb-5b7a-47d5-9ca7-96f9d5ec798d None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:50.347753 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 453/912] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:50.353052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290fff3-4719-42f9-82e3-85ab7936c427 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:50.353608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290fff3-4719-42f9-82e3-85ab7936c427 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:50.354084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290fff3-4719-42f9-82e3-85ab7936c427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:50.359936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a1c59-8aa5-422d-8b8c-8a58b27fd167 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:50.371321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7290fff3-4719-42f9-82e3-85ab7936c427 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:50.372859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7290fff3-4719-42f9-82e3-85ab7936c427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:50.395673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58e5f9-b46a-4af9-bc79-1c6704787f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:50.396264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58e5f9-b46a-4af9-bc79-1c6704787f02 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:50.396585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58e5f9-b46a-4af9-bc79-1c6704787f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:50.396860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58e5f9-b46a-4af9-bc79-1c6704787f02 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:50.397596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58e5f9-b46a-4af9-bc79-1c6704787f02 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:50.400236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58e5f9-b46a-4af9-bc79-1c6704787f02 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:50.415879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 460/913] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:21:50 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:50.460142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 454/914] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:50.483957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58e5f9-b46a-4af9-bc79-1c6704787f02 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:50.598904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58e5f9-b46a-4af9-bc79-1c6704787f02 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:50.626916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:50.639334 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3148d8f-45cb-4ccc-927b-595ae16979f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:50.640141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3148d8f-45cb-4ccc-927b-595ae16979f2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:50.640459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3148d8f-45cb-4ccc-927b-595ae16979f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:50.640733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3148d8f-45cb-4ccc-927b-595ae16979f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:50.653795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3148d8f-45cb-4ccc-927b-595ae16979f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-c3148d8f-45cb-4ccc-927b-595ae16979f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:50.661986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:50.662895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:50.663738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 461/915] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jul 21 17:21:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:50.669552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290fff3-4719-42f9-82e3-85ab7936c427 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:50.739154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:50.778417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290fff3-4719-42f9-82e3-85ab7936c427 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:50.813059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33085260-5897-44aa-89da-455a6053885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:50.813735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33085260-5897-44aa-89da-455a6053885b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:50.814014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33085260-5897-44aa-89da-455a6053885b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:50.814284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33085260-5897-44aa-89da-455a6053885b None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:50.815024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33085260-5897-44aa-89da-455a6053885b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:50.815326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33085260-5897-44aa-89da-455a6053885b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:50.816713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33085260-5897-44aa-89da-455a6053885b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:50.826065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 455/916] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jul 21 17:21:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:50.831702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3148d8f-45cb-4ccc-927b-595ae16979f2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:50.936621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3148d8f-45cb-4ccc-927b-595ae16979f2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:50.972443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923d93b-7452-4c84-ad23-dbcb4b2074b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:50.973240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923d93b-7452-4c84-ad23-dbcb4b2074b8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:50.973519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923d93b-7452-4c84-ad23-dbcb4b2074b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:50.973791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923d93b-7452-4c84-ad23-dbcb4b2074b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:50.979010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf104e-8294-431f-a56f-2dca63fa6994 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:50.979301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf104e-8294-431f-a56f-2dca63fa6994 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:50.979582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf104e-8294-431f-a56f-2dca63fa6994 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:50.984841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 456/917] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:50.985803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923d93b-7452-4c84-ad23-dbcb4b2074b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:50.986445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db5cb8-ab26-48b5-8e6a-0d3419988d49 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:50.986735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db5cb8-ab26-48b5-8e6a-0d3419988d49 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:50.987014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db5cb8-ab26-48b5-8e6a-0d3419988d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:50.987262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923d93b-7452-4c84-ad23-dbcb4b2074b8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:51.003933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-06db5cb8-ab26-48b5-8e6a-0d3419988d49 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:51.005150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-06db5cb8-ab26-48b5-8e6a-0d3419988d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-4923d93b-7452-4c84-ad23-dbcb4b2074b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:51.005333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:51.006174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:51.007063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:51.007063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:51.007063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:51.007063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:51.007063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 462/918] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:51.013363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923d93b-7452-4c84-ad23-dbcb4b2074b8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:51.057458 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:51.080688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 457/919] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:51.104128 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db5cb8-ab26-48b5-8e6a-0d3419988d49 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:51.130036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923d93b-7452-4c84-ad23-dbcb4b2074b8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:51.169271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e6171-515e-4f12-a01a-f0a281d0ad7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:51.169941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e6171-515e-4f12-a01a-f0a281d0ad7a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:51.170226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e6171-515e-4f12-a01a-f0a281d0ad7a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:51.170535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e6171-515e-4f12-a01a-f0a281d0ad7a None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:51.171362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e6171-515e-4f12-a01a-f0a281d0ad7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:51.171672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e6171-515e-4f12-a01a-f0a281d0ad7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:51.173092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e6171-515e-4f12-a01a-f0a281d0ad7a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:51.181417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:51.190526 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 463/920] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:51.191860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e6171-515e-4f12-a01a-f0a281d0ad7a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:51.235568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db5cb8-ab26-48b5-8e6a-0d3419988d49 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:51.271899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ad574-6424-47d9-99ef-5e06b2168631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:51.272898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ad574-6424-47d9-99ef-5e06b2168631 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:51.273191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ad574-6424-47d9-99ef-5e06b2168631 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:51.274032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ad574-6424-47d9-99ef-5e06b2168631 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:51.275189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ad574-6424-47d9-99ef-5e06b2168631 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:51.276877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ad574-6424-47d9-99ef-5e06b2168631 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-4f4ad574-6424-47d9-99ef-5e06b2168631 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:51.293296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:51.294301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:51.300449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 458/921] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:21:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:51.301254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ad574-6424-47d9-99ef-5e06b2168631 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:51.330636 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:51.336675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e6171-515e-4f12-a01a-f0a281d0ad7a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:51.374830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a55caa-d666-4f11-86c0-3fc8b7a226af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:51.375642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55caa-d666-4f11-86c0-3fc8b7a226af None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:51.375924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55caa-d666-4f11-86c0-3fc8b7a226af None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:51.376199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55caa-d666-4f11-86c0-3fc8b7a226af None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:51.384232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a55caa-d666-4f11-86c0-3fc8b7a226af None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:51.385772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a55caa-d666-4f11-86c0-3fc8b7a226af None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-80a55caa-d666-4f11-86c0-3fc8b7a226af None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:51.401107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:51.402455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:51.403386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:51.403386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:51.403386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:51.403386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:51.403386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:51.403386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:51.403386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:51.403386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:51.403386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:51.403386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:51.407429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 464/922] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jul 21 17:21:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:51.408771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a55caa-d666-4f11-86c0-3fc8b7a226af None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:51.427639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ad574-6424-47d9-99ef-5e06b2168631 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:51.449733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:51.468038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4639607-7b98-4893-a9ce-5d82317ca4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:51.469036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4639607-7b98-4893-a9ce-5d82317ca4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:51.469518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4639607-7b98-4893-a9ce-5d82317ca4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:51.470158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4639607-7b98-4893-a9ce-5d82317ca4b3 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:51.471155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4639607-7b98-4893-a9ce-5d82317ca4b3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:51.471681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4639607-7b98-4893-a9ce-5d82317ca4b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:51.473360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4639607-7b98-4893-a9ce-5d82317ca4b3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:51.484256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 459/923] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:21:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:51.485377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4639607-7b98-4893-a9ce-5d82317ca4b3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:51.513967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a55caa-d666-4f11-86c0-3fc8b7a226af None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:51.550805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbde71-5dbe-4f96-a5ed-c92f812626a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:51.551484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbde71-5dbe-4f96-a5ed-c92f812626a1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:51.551771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbde71-5dbe-4f96-a5ed-c92f812626a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:51.552134 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbde71-5dbe-4f96-a5ed-c92f812626a1 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:51.552990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbde71-5dbe-4f96-a5ed-c92f812626a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:51.553351 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbde71-5dbe-4f96-a5ed-c92f812626a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:51.554894 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbde71-5dbe-4f96-a5ed-c92f812626a1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:51.577596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 465/924] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jul 21 17:21:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:51.579019 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbde71-5dbe-4f96-a5ed-c92f812626a1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:51.596599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4639607-7b98-4893-a9ce-5d82317ca4b3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:51.598312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:51.640274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459af49-81c6-4dfa-a306-9afb3c8d16bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:51.641392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459af49-81c6-4dfa-a306-9afb3c8d16bb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:51.642035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459af49-81c6-4dfa-a306-9afb3c8d16bb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:51.642599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459af49-81c6-4dfa-a306-9afb3c8d16bb None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:51.643527 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8459af49-81c6-4dfa-a306-9afb3c8d16bb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:51.644028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8459af49-81c6-4dfa-a306-9afb3c8d16bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:51.645879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8459af49-81c6-4dfa-a306-9afb3c8d16bb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:51.657191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 460/925] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Jul 21 17:21:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:51.658418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459af49-81c6-4dfa-a306-9afb3c8d16bb None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:51.689365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbde71-5dbe-4f96-a5ed-c92f812626a1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:51.713385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:51.726986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ade0b-7eb4-4328-8f0a-d74e4dd359a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:51.727941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ade0b-7eb4-4328-8f0a-d74e4dd359a4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:51.728257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ade0b-7eb4-4328-8f0a-d74e4dd359a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:51.728567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ade0b-7eb4-4328-8f0a-d74e4dd359a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:51.739201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ade0b-7eb4-4328-8f0a-d74e4dd359a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:51.740827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ade0b-7eb4-4328-8f0a-d74e4dd359a4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-715ade0b-7eb4-4328-8f0a-d74e4dd359a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:51.759862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:51.761069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:51.761944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:51.761944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:51.761944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:51.761944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:51.766269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 466/926] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:51.767667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ade0b-7eb4-4328-8f0a-d74e4dd359a4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:51.776942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459af49-81c6-4dfa-a306-9afb3c8d16bb None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:51.813688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce817e-cb46-490b-833c-22977583fabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:51.814800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce817e-cb46-490b-833c-22977583fabc None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:51.815282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce817e-cb46-490b-833c-22977583fabc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:51.815755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce817e-cb46-490b-833c-22977583fabc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:51.816817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce817e-cb46-490b-833c-22977583fabc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:51.818479 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce817e-cb46-490b-833c-22977583fabc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:51.841004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:51.843826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 461/927] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jul 21 17:21:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:51.890163 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ade0b-7eb4-4328-8f0a-d74e4dd359a4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:51.940828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c14761-798d-4bba-ae15-b4208100cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:51.941570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c14761-798d-4bba-ae15-b4208100cb9f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:51.942058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c14761-798d-4bba-ae15-b4208100cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:51.942342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c14761-798d-4bba-ae15-b4208100cb9f None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:51.943128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c14761-798d-4bba-ae15-b4208100cb9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:51.943444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c14761-798d-4bba-ae15-b4208100cb9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:51.944878 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c14761-798d-4bba-ae15-b4208100cb9f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:51.964513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 467/928] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:51.966257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce817e-cb46-490b-833c-22977583fabc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:52.082488 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce817e-cb46-490b-833c-22977583fabc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:52.086939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:52.093027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cdf6f-f4c6-4454-bb47-f838c26974b4 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.093316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cdf6f-f4c6-4454-bb47-f838c26974b4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.093595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cdf6f-f4c6-4454-bb47-f838c26974b4 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.095203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 468/929] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:52.101174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749a0d8-2333-47b4-8fa8-acea53c8622f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.102384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749a0d8-2333-47b4-8fa8-acea53c8622f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.102932 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749a0d8-2333-47b4-8fa8-acea53c8622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.122144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c8fb9-3900-4f8c-b63f-07c0010bcfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:52.122942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c8fb9-3900-4f8c-b63f-07c0010bcfe9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.123048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2749a0d8-2333-47b4-8fa8-acea53c8622f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:52.123210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c8fb9-3900-4f8c-b63f-07c0010bcfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.123482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c8fb9-3900-4f8c-b63f-07c0010bcfe9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.124186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2749a0d8-2333-47b4-8fa8-acea53c8622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:52.131342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c8fb9-3900-4f8c-b63f-07c0010bcfe9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:52.132877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c8fb9-3900-4f8c-b63f-07c0010bcfe9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-7f9c8fb9-3900-4f8c-b63f-07c0010bcfe9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:52.147321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:52.148853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:52.150273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:52.150273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:52.150273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:52.150273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:52.150273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:52.150273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:52.150273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:52.150273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:52.150273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 462/930] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:52.154904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c8fb9-3900-4f8c-b63f-07c0010bcfe9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:52.212333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 469/931] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:52.244025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749a0d8-2333-47b4-8fa8-acea53c8622f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:52.285610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c8fb9-3900-4f8c-b63f-07c0010bcfe9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:52.312944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:52.322429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01090398-b985-4a2c-9902-862c45edada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:52.323711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01090398-b985-4a2c-9902-862c45edada9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.324210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01090398-b985-4a2c-9902-862c45edada9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.324746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01090398-b985-4a2c-9902-862c45edada9 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.325788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01090398-b985-4a2c-9902-862c45edada9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:52.326301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01090398-b985-4a2c-9902-862c45edada9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:52.327966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01090398-b985-4a2c-9902-862c45edada9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:52.347365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 463/932] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:52.348985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01090398-b985-4a2c-9902-862c45edada9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:52.366756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749a0d8-2333-47b4-8fa8-acea53c8622f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:52.377207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:52.410975 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b0d33-d96c-4d42-adf5-ae903849160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:52.411787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b0d33-d96c-4d42-adf5-ae903849160c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.412065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b0d33-d96c-4d42-adf5-ae903849160c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.412356 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b0d33-d96c-4d42-adf5-ae903849160c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.413018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b0d33-d96c-4d42-adf5-ae903849160c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:52.415206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b0d33-d96c-4d42-adf5-ae903849160c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-328b0d33-d96c-4d42-adf5-ae903849160c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:52.431565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:52.432614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:52.438616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 470/933] 10.222.0.57 () {56 vars in 1132 bytes} [Thu Jul 21 17:21:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:52.441162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b0d33-d96c-4d42-adf5-ae903849160c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:52.476566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01090398-b985-4a2c-9902-862c45edada9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:52.514554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd00d7-34dc-4b06-ab63-0bc325602192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:52.515852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd00d7-34dc-4b06-ab63-0bc325602192 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.516425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd00d7-34dc-4b06-ab63-0bc325602192 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.516953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd00d7-34dc-4b06-ab63-0bc325602192 None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/75f11eaa4b504d0aa4018fb4ffbbd3a2/roles/085ea69bad9746939d7f6aad1904b3f3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.519511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd00d7-34dc-4b06-ab63-0bc325602192 None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=75f11eaa4b504d0aa4018fb4ffbbd3a2, role_id=085ea69bad9746939d7f6aad1904b3f3)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:52.521176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd00d7-34dc-4b06-ab63-0bc325602192 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:52.544768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 464/934] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:21:52 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/75f11eaa4b504d0aa4018fb4ffbbd3a2/roles/085ea69bad9746939d7f6aad1904b3f3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:21:52.559811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b0d33-d96c-4d42-adf5-ae903849160c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:52.580400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:52.584052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f410f5a-96b2-42f6-9018-11ecfc5ebeff None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.584613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f410f5a-96b2-42f6-9018-11ecfc5ebeff None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.585097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f410f5a-96b2-42f6-9018-11ecfc5ebeff None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.588529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 465/935] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:52.594134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521944e5-f19a-4bd8-a3d1-10411b4a4c41 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.594835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521944e5-f19a-4bd8-a3d1-10411b4a4c41 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.595334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521944e5-f19a-4bd8-a3d1-10411b4a4c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.606514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e41cc9-d4f7-4a99-90e5-95db7c34fbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:52.607206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41cc9-d4f7-4a99-90e5-95db7c34fbbf None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.607497 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41cc9-d4f7-4a99-90e5-95db7c34fbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.607776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41cc9-d4f7-4a99-90e5-95db7c34fbbf None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.608552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e41cc9-d4f7-4a99-90e5-95db7c34fbbf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:52.609154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e41cc9-d4f7-4a99-90e5-95db7c34fbbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:52.610318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e41cc9-d4f7-4a99-90e5-95db7c34fbbf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:52.616814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 471/936] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:21:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:52.622983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e41cc9-d4f7-4a99-90e5-95db7c34fbbf None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:52.623464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-521944e5-f19a-4bd8-a3d1-10411b4a4c41 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:52.625248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-521944e5-f19a-4bd8-a3d1-10411b4a4c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:52.708481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 466/937] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:52.731788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521944e5-f19a-4bd8-a3d1-10411b4a4c41 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:52.739079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e41cc9-d4f7-4a99-90e5-95db7c34fbbf None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:52.748705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:52.783023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d0f68-af3f-4ee4-a3c1-b8644a375e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:52.783916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d0f68-af3f-4ee4-a3c1-b8644a375e22 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.784473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d0f68-af3f-4ee4-a3c1-b8644a375e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.784770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d0f68-af3f-4ee4-a3c1-b8644a375e22 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.785483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d0f68-af3f-4ee4-a3c1-b8644a375e22 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:52.785789 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d0f68-af3f-4ee4-a3c1-b8644a375e22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:52.787698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d0f68-af3f-4ee4-a3c1-b8644a375e22 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:52.794312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 472/938] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:21:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:52.799686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d0f68-af3f-4ee4-a3c1-b8644a375e22 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:52.858064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521944e5-f19a-4bd8-a3d1-10411b4a4c41 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:52.863230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:52.900864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01149904-9ef5-4195-8ce6-0bcec508bf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:52.901980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01149904-9ef5-4195-8ce6-0bcec508bf6d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.902468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01149904-9ef5-4195-8ce6-0bcec508bf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.902923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01149904-9ef5-4195-8ce6-0bcec508bf6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.903864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01149904-9ef5-4195-8ce6-0bcec508bf6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:52.905428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01149904-9ef5-4195-8ce6-0bcec508bf6d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-01149904-9ef5-4195-8ce6-0bcec508bf6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:52.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:52.922376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:52.923219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:52.925776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 467/939] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:21:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:52.932379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01149904-9ef5-4195-8ce6-0bcec508bf6d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:52.940221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d0f68-af3f-4ee4-a3c1-b8644a375e22 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:52.977788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625c51d-1143-455b-8b58-63d576dbad43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:52.978822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625c51d-1143-455b-8b58-63d576dbad43 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:52.979324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625c51d-1143-455b-8b58-63d576dbad43 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:52.979800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625c51d-1143-455b-8b58-63d576dbad43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:52.980848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625c51d-1143-455b-8b58-63d576dbad43 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:52.982442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625c51d-1143-455b-8b58-63d576dbad43 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:53.008648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 473/940] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jul 21 17:21:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:53.027082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:53.055698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01149904-9ef5-4195-8ce6-0bcec508bf6d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:53.104027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece5f3c-6745-4f01-ab52-5b774106cf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:53.105116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece5f3c-6745-4f01-ab52-5b774106cf28 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:53.105640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece5f3c-6745-4f01-ab52-5b774106cf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:53.106126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece5f3c-6745-4f01-ab52-5b774106cf28 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:53.107081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece5f3c-6745-4f01-ab52-5b774106cf28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:53.107636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece5f3c-6745-4f01-ab52-5b774106cf28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:53.109347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece5f3c-6745-4f01-ab52-5b774106cf28 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:53.116001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 468/941] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jul 21 17:21:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:53.123094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625c51d-1143-455b-8b58-63d576dbad43 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:53.234465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625c51d-1143-455b-8b58-63d576dbad43 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:53.236732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:53.272753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4554bf6-c150-4213-a805-fd60ab370ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:53.274007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4554bf6-c150-4213-a805-fd60ab370ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:53.274279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4554bf6-c150-4213-a805-fd60ab370ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:53.274545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4554bf6-c150-4213-a805-fd60ab370ef9 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:53.275253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4554bf6-c150-4213-a805-fd60ab370ef9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:53.275557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4554bf6-c150-4213-a805-fd60ab370ef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:53.276956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4554bf6-c150-4213-a805-fd60ab370ef9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:53.282310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 474/942] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jul 21 17:21:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:53.288204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece5f3c-6745-4f01-ab52-5b774106cf28 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:53.395600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece5f3c-6745-4f01-ab52-5b774106cf28 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:53.431368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ffeba-aacf-4e48-ab0d-edb296265a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:53.432085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ffeba-aacf-4e48-ab0d-edb296265a5d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:53.432399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ffeba-aacf-4e48-ab0d-edb296265a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:53.432681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ffeba-aacf-4e48-ab0d-edb296265a5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:53.433472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ffeba-aacf-4e48-ab0d-edb296265a5d None admin] RBAC: Adding query filter params (interface=public, service_id=c86170e37a6a4a089e80f9ab3c7f77a2, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:53.433786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ffeba-aacf-4e48-ab0d-edb296265a5d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:53.435153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ffeba-aacf-4e48-ab0d-edb296265a5d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:53.440718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 469/943] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jul 21 17:21:53 2022] GET /identity/v3/endpoints?service_id=c86170e37a6a4a089e80f9ab3c7f77a2&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:53.446982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4554bf6-c150-4213-a805-fd60ab370ef9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:53.556331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4554bf6-c150-4213-a805-fd60ab370ef9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:53.592218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98701634-7d31-45d9-9fa8-596e64395608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:53.593349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98701634-7d31-45d9-9fa8-596e64395608 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:53.593897 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98701634-7d31-45d9-9fa8-596e64395608 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:53.594383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98701634-7d31-45d9-9fa8-596e64395608 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:53.595272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98701634-7d31-45d9-9fa8-596e64395608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:53.596882 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98701634-7d31-45d9-9fa8-596e64395608 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:53.603759 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 475/944] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jul 21 17:21:53 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:21:54.069491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8a254-04c8-44fb-b07e-7a1ffa4f92a8 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:54.069887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8a254-04c8-44fb-b07e-7a1ffa4f92a8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:54.070132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8a254-04c8-44fb-b07e-7a1ffa4f92a8 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:54.071724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 470/945] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:54.077445 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a48bf-a2ec-41c6-b7fb-52b1db2555b7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:54.077736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a48bf-a2ec-41c6-b7fb-52b1db2555b7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:54.078055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a48bf-a2ec-41c6-b7fb-52b1db2555b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:54.095301 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0c9a48bf-a2ec-41c6-b7fb-52b1db2555b7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:54.096509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0c9a48bf-a2ec-41c6-b7fb-52b1db2555b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:54.121888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:54.165954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 476/946] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:54.191176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8a254-04c8-44fb-b07e-7a1ffa4f92a8 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:54.194289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:54.305454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8a254-04c8-44fb-b07e-7a1ffa4f92a8 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:54.342130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:54.343076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:54.343519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:54.343951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:54.344844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:54.346426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:54.347951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: WARNING keystone.server.flask.common [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 17:21:54.366993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.sql.core [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 17:21:54.366993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 17:21:54.366993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [parameters: {'id': '32fca33d3a684cceaf06f4f687dcf0ff', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 17:21:54.366993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80211) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 32fca33d3a684cceaf06f4f687dcf0ff.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 32fca33d3a684cceaf06f4f687dcf0ff. Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application conn.query(q) Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 17:21:54.384959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result.read() Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 17:21:54.386034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application next(self.gen) Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application yield resource Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application next(self.gen) Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise self.value Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application session.commit() Jul 21 17:21:54.387245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.session.flush() Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application compat.raise_( Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 17:21:54.388251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 17:21:54.389167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application conn.query(q) Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result.read() Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 17:21:54.390104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [parameters: {'id': '32fca33d3a684cceaf06f4f687dcf0ff', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:54.390973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:54.391797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:54.392792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:54.392792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 17:21:54.392792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 17:21:54.392792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:54.392792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:54.392792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 17:21:54.392792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 17:21:54.392792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 32fca33d3a684cceaf06f4f687dcf0ff. Jul 21 17:21:54.392792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:54.395211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 471/947] 10.222.0.57 () {60 vars in 1140 bytes} [Thu Jul 21 17:21:54 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:54.401398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a48bf-a2ec-41c6-b7fb-52b1db2555b7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:54.506486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a48bf-a2ec-41c6-b7fb-52b1db2555b7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:54.542972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d6153-cc22-4341-b42a-95b04be12631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:54.543767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d6153-cc22-4341-b42a-95b04be12631 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:54.544046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d6153-cc22-4341-b42a-95b04be12631 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:54.544334 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d6153-cc22-4341-b42a-95b04be12631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:54.551372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d6153-cc22-4341-b42a-95b04be12631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:54.552852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d6153-cc22-4341-b42a-95b04be12631 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-585d6153-cc22-4341-b42a-95b04be12631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:54.567080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:54.567999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:54.568976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:54.568976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:54.568976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:54.568976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:54.568976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:54.568976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:54.568976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:54.568976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:54.568976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 477/948] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:54.574569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:54.641479 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:54.692319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85899cd8-7b43-433a-af75-b66e7d5d4e4a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:54.692319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85899cd8-7b43-433a-af75-b66e7d5d4e4a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:54.692319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85899cd8-7b43-433a-af75-b66e7d5d4e4a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:54.692319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 478/949] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:54.693575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001a553-e32c-4412-afd9-151281fe739a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:54.693940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4e569-91bb-47b1-a6ce-6ea38c50168c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:54.694234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4e569-91bb-47b1-a6ce-6ea38c50168c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:54.694517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4e569-91bb-47b1-a6ce-6ea38c50168c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:54.713429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1df4e569-91bb-47b1-a6ce-6ea38c50168c None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:54.714622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1df4e569-91bb-47b1-a6ce-6ea38c50168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:54.730158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df343afa-9bb3-4371-9eaa-a93f4a738daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:54.730823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df343afa-9bb3-4371-9eaa-a93f4a738daf None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:54.731108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df343afa-9bb3-4371-9eaa-a93f4a738daf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:54.731351 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:54.731483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df343afa-9bb3-4371-9eaa-a93f4a738daf None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:54.732253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df343afa-9bb3-4371-9eaa-a93f4a738daf None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:54.732464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df343afa-9bb3-4371-9eaa-a93f4a738daf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:54.734188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df343afa-9bb3-4371-9eaa-a93f4a738daf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:54.755055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 472/950] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:21:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:54.797595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 479/951] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:54.822326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df343afa-9bb3-4371-9eaa-a93f4a738daf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:54.940651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df343afa-9bb3-4371-9eaa-a93f4a738daf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:54.960562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:54.979777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e2cf6-71a9-42e9-afb9-2f91f30f28b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:54.980903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e2cf6-71a9-42e9-afb9-2f91f30f28b0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:54.981343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e2cf6-71a9-42e9-afb9-2f91f30f28b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:54.981768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e2cf6-71a9-42e9-afb9-2f91f30f28b0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:54.993115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e2cf6-71a9-42e9-afb9-2f91f30f28b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e41e2cf6-71a9-42e9-afb9-2f91f30f28b0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:21:55.001433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:55.002556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:21:55.003577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:55.005941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 473/952] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jul 21 17:21:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:55.012770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4e569-91bb-47b1-a6ce-6ea38c50168c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:55.123027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4e569-91bb-47b1-a6ce-6ea38c50168c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:55.159343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76d105-3c53-444a-bc10-e793318073f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:55.160023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76d105-3c53-444a-bc10-e793318073f2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.160337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76d105-3c53-444a-bc10-e793318073f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.160603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76d105-3c53-444a-bc10-e793318073f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.161350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76d105-3c53-444a-bc10-e793318073f2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:55.161655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76d105-3c53-444a-bc10-e793318073f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:55.163021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76d105-3c53-444a-bc10-e793318073f2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:55.176406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 480/953] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jul 21 17:21:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:55.178078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e2cf6-71a9-42e9-afb9-2f91f30f28b0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:55.242899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784ca0c-b6c3-4e58-b24f-410cd6cd8471 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.243550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784ca0c-b6c3-4e58-b24f-410cd6cd8471 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.243841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784ca0c-b6c3-4e58-b24f-410cd6cd8471 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.246223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 481/954] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:55.251424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218b2e9-0dad-438a-9c56-f7003d08898b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.251710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218b2e9-0dad-438a-9c56-f7003d08898b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.251983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218b2e9-0dad-438a-9c56-f7003d08898b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.270364 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b218b2e9-0dad-438a-9c56-f7003d08898b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:55.271553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b218b2e9-0dad-438a-9c56-f7003d08898b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:55.280414 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:55.294288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e2cf6-71a9-42e9-afb9-2f91f30f28b0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:55.330016 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ad0bb-7132-4476-9012-09090f03ef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:55.330803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ad0bb-7132-4476-9012-09090f03ef1f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.331088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ad0bb-7132-4476-9012-09090f03ef1f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.331361 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ad0bb-7132-4476-9012-09090f03ef1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.342471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ad0bb-7132-4476-9012-09090f03ef1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:55.344216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ad0bb-7132-4476-9012-09090f03ef1f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:55.349408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 482/955] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-302ad0bb-7132-4476-9012-09090f03ef1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:55.367716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:55.368776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:55.369830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:55.369830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:55.369830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:55.369830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:55.371656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218b2e9-0dad-438a-9c56-f7003d08898b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:55.372395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 474/956] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:55.375043 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ad0bb-7132-4476-9012-09090f03ef1f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:55.416541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:55.481678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218b2e9-0dad-438a-9c56-f7003d08898b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:55.513215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ad0bb-7132-4476-9012-09090f03ef1f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:55.517546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ec8d4-18d2-4ff4-adc3-46f9a7aaf5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:55.518336 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ec8d4-18d2-4ff4-adc3-46f9a7aaf5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.518617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ec8d4-18d2-4ff4-adc3-46f9a7aaf5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.518893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ec8d4-18d2-4ff4-adc3-46f9a7aaf5e8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.519547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ec8d4-18d2-4ff4-adc3-46f9a7aaf5e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:55.520969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ec8d4-18d2-4ff4-adc3-46f9a7aaf5e8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-155ec8d4-18d2-4ff4-adc3-46f9a7aaf5e8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:55.532982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:55.534094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 17:21:55.534950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:55.539887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 483/957] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:55.540787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ec8d4-18d2-4ff4-adc3-46f9a7aaf5e8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:55.550536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:55.560639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e4c5c-cce9-4459-a99e-335cc5bf222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:55.561799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e4c5c-cce9-4459-a99e-335cc5bf222d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.562463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e4c5c-cce9-4459-a99e-335cc5bf222d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.562963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e4c5c-cce9-4459-a99e-335cc5bf222d None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.564172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e4c5c-cce9-4459-a99e-335cc5bf222d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:55.564709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e4c5c-cce9-4459-a99e-335cc5bf222d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:55.566741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e4c5c-cce9-4459-a99e-335cc5bf222d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:55.583309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 475/958] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:55.589391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e4c5c-cce9-4459-a99e-335cc5bf222d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:55.654542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ec8d4-18d2-4ff4-adc3-46f9a7aaf5e8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:55.691798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:55.694453 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94939fd4-30cd-4273-8a91-cc5886825666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:55.695504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94939fd4-30cd-4273-8a91-cc5886825666 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.696689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94939fd4-30cd-4273-8a91-cc5886825666 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.697279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94939fd4-30cd-4273-8a91-cc5886825666 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.698189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94939fd4-30cd-4273-8a91-cc5886825666 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:55.698959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94939fd4-30cd-4273-8a91-cc5886825666 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:55.700823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94939fd4-30cd-4273-8a91-cc5886825666 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:55.703158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e4c5c-cce9-4459-a99e-335cc5bf222d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:55.706970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 484/959] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:21:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:55.714249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94939fd4-30cd-4273-8a91-cc5886825666 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:55.741221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de215ae-ff8a-44d8-a286-1a28990a3dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:55.742668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de215ae-ff8a-44d8-a286-1a28990a3dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.743155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de215ae-ff8a-44d8-a286-1a28990a3dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.743637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de215ae-ff8a-44d8-a286-1a28990a3dd5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.752588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de215ae-ff8a-44d8-a286-1a28990a3dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:55.754003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de215ae-ff8a-44d8-a286-1a28990a3dd5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-0de215ae-ff8a-44d8-a286-1a28990a3dd5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:55.769845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:55.770855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:55.771835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 476/960] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jul 21 17:21:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:55.777542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de215ae-ff8a-44d8-a286-1a28990a3dd5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:55.800398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:55.842050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94939fd4-30cd-4273-8a91-cc5886825666 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:55.878329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97692630-deff-40ff-8877-c6442a606a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:55.878990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97692630-deff-40ff-8877-c6442a606a7e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.879278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97692630-deff-40ff-8877-c6442a606a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.879552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97692630-deff-40ff-8877-c6442a606a7e None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.880253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97692630-deff-40ff-8877-c6442a606a7e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:55.880609 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97692630-deff-40ff-8877-c6442a606a7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:55.881944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97692630-deff-40ff-8877-c6442a606a7e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:55.896332 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 485/961] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:21:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:55.896537 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de215ae-ff8a-44d8-a286-1a28990a3dd5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:55.896654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0ee27-977c-479f-be64-6aa7e58a2386 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.896654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0ee27-977c-479f-be64-6aa7e58a2386 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.896654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0ee27-977c-479f-be64-6aa7e58a2386 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.896654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 486/962] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:55.896654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd7c64-3f50-4b3f-bbfc-914dd676cc27 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.896654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd7c64-3f50-4b3f-bbfc-914dd676cc27 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.897113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd7c64-3f50-4b3f-bbfc-914dd676cc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.921519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c0fd7c64-3f50-4b3f-bbfc-914dd676cc27 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:55.922672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c0fd7c64-3f50-4b3f-bbfc-914dd676cc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:55.942228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f636db3-7e32-47ad-955b-3379e787ee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:55.943333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f636db3-7e32-47ad-955b-3379e787ee00 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:55.943812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f636db3-7e32-47ad-955b-3379e787ee00 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:55.944276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f636db3-7e32-47ad-955b-3379e787ee00 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:55.945647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f636db3-7e32-47ad-955b-3379e787ee00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:55.946118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f636db3-7e32-47ad-955b-3379e787ee00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:55.947816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f636db3-7e32-47ad-955b-3379e787ee00 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:55.982483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 477/963] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Jul 21 17:21:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:55.986452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f636db3-7e32-47ad-955b-3379e787ee00 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:56.005444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:56.018600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 487/964] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:56.043060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd7c64-3f50-4b3f-bbfc-914dd676cc27 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:56.116133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:56.127941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f636db3-7e32-47ad-955b-3379e787ee00 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:56.157212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd7c64-3f50-4b3f-bbfc-914dd676cc27 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:56.175897 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bd413-494e-426c-a349-170956d8da22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:56.176805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bd413-494e-426c-a349-170956d8da22 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:56.177094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bd413-494e-426c-a349-170956d8da22 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:56.177377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bd413-494e-426c-a349-170956d8da22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:56.197981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2ddeb-65fb-42ef-b7d0-81aeeaf0c26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:56.200463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2ddeb-65fb-42ef-b7d0-81aeeaf0c26f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:56.200744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2ddeb-65fb-42ef-b7d0-81aeeaf0c26f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:56.201017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2ddeb-65fb-42ef-b7d0-81aeeaf0c26f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:56.201668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2ddeb-65fb-42ef-b7d0-81aeeaf0c26f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:56.203035 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2ddeb-65fb-42ef-b7d0-81aeeaf0c26f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-7cb2ddeb-65fb-42ef-b7d0-81aeeaf0c26f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:56.222149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:56.223210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:56.224052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 488/965] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:21:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:56.231588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2ddeb-65fb-42ef-b7d0-81aeeaf0c26f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:56.274782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5bd413-494e-426c-a349-170956d8da22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:56.276227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5bd413-494e-426c-a349-170956d8da22 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:56.292081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:56.370530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2ddeb-65fb-42ef-b7d0-81aeeaf0c26f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:56.387111 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 478/966] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:21:55 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:21:56.394239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bd413-494e-426c-a349-170956d8da22 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:56.409270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:56.419001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85428d7-fb70-458f-b9b5-d3d6ebcce18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:56.419925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85428d7-fb70-458f-b9b5-d3d6ebcce18c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:56.420208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85428d7-fb70-458f-b9b5-d3d6ebcce18c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:56.420503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85428d7-fb70-458f-b9b5-d3d6ebcce18c None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:56.421271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85428d7-fb70-458f-b9b5-d3d6ebcce18c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:56.421582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85428d7-fb70-458f-b9b5-d3d6ebcce18c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:56.422981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85428d7-fb70-458f-b9b5-d3d6ebcce18c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:56.433611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 489/967] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jul 21 17:21:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:56.434415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85428d7-fb70-458f-b9b5-d3d6ebcce18c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:56.525669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bd413-494e-426c-a349-170956d8da22 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:56.535773 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:56.551884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85428d7-fb70-458f-b9b5-d3d6ebcce18c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:56.566162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d4d24-60f4-43e0-a37f-f79548d08b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:56.567063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d4d24-60f4-43e0-a37f-f79548d08b7b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:56.567355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d4d24-60f4-43e0-a37f-f79548d08b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:56.567631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d4d24-60f4-43e0-a37f-f79548d08b7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:56.580337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d4d24-60f4-43e0-a37f-f79548d08b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:56.581922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d4d24-60f4-43e0-a37f-f79548d08b7b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:56.588750 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9074a0-308d-443b-86d1-864125f371de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:56.589428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9074a0-308d-443b-86d1-864125f371de None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:56.589710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9074a0-308d-443b-86d1-864125f371de None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:56.589986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9074a0-308d-443b-86d1-864125f371de None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:56.590697 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9074a0-308d-443b-86d1-864125f371de None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:56.591065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9074a0-308d-443b-86d1-864125f371de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:56.592472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9074a0-308d-443b-86d1-864125f371de None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:56.597756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 490/968] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jul 21 17:21:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-4b2d4d24-60f4-43e0-a37f-f79548d08b7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:21:56.602981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:56.603926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:56.605983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:21:56.605983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:21:56.605983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:21:56.605983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:56.606473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9074a0-308d-443b-86d1-864125f371de None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:56.610092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 479/969] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:56.610092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d4d24-60f4-43e0-a37f-f79548d08b7b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:56.662571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:56.709021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9074a0-308d-443b-86d1-864125f371de None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:56.730594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d4d24-60f4-43e0-a37f-f79548d08b7b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:56.743878 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ec9ac-dd87-4d19-b360-d894bf722d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:56.744778 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ec9ac-dd87-4d19-b360-d894bf722d68 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:56.744934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ec9ac-dd87-4d19-b360-d894bf722d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:56.745175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ec9ac-dd87-4d19-b360-d894bf722d68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:56.745965 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ec9ac-dd87-4d19-b360-d894bf722d68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:56.747336 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ec9ac-dd87-4d19-b360-d894bf722d68 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:56.769859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 491/970] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jul 21 17:21:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:21:56.788529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05a2f4-0226-4ada-bf10-3c3afe081877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:56.789221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a2f4-0226-4ada-bf10-3c3afe081877 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:56.789560 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a2f4-0226-4ada-bf10-3c3afe081877 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:56.789845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a2f4-0226-4ada-bf10-3c3afe081877 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:56.790587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05a2f4-0226-4ada-bf10-3c3afe081877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:56.790898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05a2f4-0226-4ada-bf10-3c3afe081877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:56.792463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05a2f4-0226-4ada-bf10-3c3afe081877 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:56.807721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 480/971] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:56.813826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ec9ac-dd87-4d19-b360-d894bf722d68 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:56.937505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ec9ac-dd87-4d19-b360-d894bf722d68 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:56.953721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:56.980262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fbab4-b11c-4d51-bec2-d3c0096feecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:56.981425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fbab4-b11c-4d51-bec2-d3c0096feecd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:56.981976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fbab4-b11c-4d51-bec2-d3c0096feecd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:56.982469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fbab4-b11c-4d51-bec2-d3c0096feecd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:56.991314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e861d7-3d66-4d40-b90c-6c410136a493 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:56.991616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e861d7-3d66-4d40-b90c-6c410136a493 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:56.991945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e861d7-3d66-4d40-b90c-6c410136a493 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:56.992744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fbab4-b11c-4d51-bec2-d3c0096feecd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:56.993619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 481/972] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:56.994728 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fbab4-b11c-4d51-bec2-d3c0096feecd None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:56.998617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215fcc1-f987-442b-90a3-98e6a1274f1e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:56.998902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215fcc1-f987-442b-90a3-98e6a1274f1e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:56.999179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215fcc1-f987-442b-90a3-98e6a1274f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-dd7fbab4-b11c-4d51-bec2-d3c0096feecd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:21:57.012003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:57.013123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:57.014247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:21:57.014247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:21:57.014247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:21:57.014247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:21:57.014247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:21:57.014247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:21:57.014247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:21:57.014247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:57.016396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 492/973] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:21:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:57.018068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3215fcc1-f987-442b-90a3-98e6a1274f1e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:57.019637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3215fcc1-f987-442b-90a3-98e6a1274f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:57.021956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fbab4-b11c-4d51-bec2-d3c0096feecd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:57.034077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:57.108870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 482/974] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:57.133339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215fcc1-f987-442b-90a3-98e6a1274f1e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:57.153963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fbab4-b11c-4d51-bec2-d3c0096feecd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:57.203081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6fbc1-e517-4023-9408-493b8d01e415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:57.203749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6fbc1-e517-4023-9408-493b8d01e415 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:57.204030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6fbc1-e517-4023-9408-493b8d01e415 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:57.204324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6fbc1-e517-4023-9408-493b8d01e415 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:57.205125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6fbc1-e517-4023-9408-493b8d01e415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:57.205435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6fbc1-e517-4023-9408-493b8d01e415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:57.206866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6fbc1-e517-4023-9408-493b8d01e415 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:57.219872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 493/975] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:21:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:57.225990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6fbc1-e517-4023-9408-493b8d01e415 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:57.261025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215fcc1-f987-442b-90a3-98e6a1274f1e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:57.265953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:57.300270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb606703-6070-42aa-ab9a-2e33d518ba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:57.301002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb606703-6070-42aa-ab9a-2e33d518ba72 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:57.301411 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb606703-6070-42aa-ab9a-2e33d518ba72 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:57.301690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb606703-6070-42aa-ab9a-2e33d518ba72 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:57.302447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb606703-6070-42aa-ab9a-2e33d518ba72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:57.304068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb606703-6070-42aa-ab9a-2e33d518ba72 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:57.318737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.sql.core [None req-fb606703-6070-42aa-ab9a-2e33d518ba72 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 17:21:57.318737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 17:21:57.318737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [parameters: {'user_id': 'b433dadc794d486d8f7af9017427915d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 17:21:57.318737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80211) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-fb606703-6070-42aa-ab9a-2e33d518ba72 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application conn.query(q) Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result.read() Jul 21 17:21:57.327711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 17:21:57.329593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application next(self.gen) Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application yield resource Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application next(self.gen) Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise self.value Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application session.commit() Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 17:21:57.330822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.session.flush() Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application compat.raise_( Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 17:21:57.331997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 17:21:57.333154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application conn.query(q) Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result.read() Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 17:21:57.334670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b433dadc794d486d8f7af9017427915d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:57.337057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 17:21:57.337947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 17:21:57.338882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 17:21:57.338882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 17:21:57.338882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 17:21:57.338882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 17:21:57.338882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 17:21:57.338882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:57.338882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 483/976] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Jul 21 17:21:57 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:21:57.338882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb606703-6070-42aa-ab9a-2e33d518ba72 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:57.350915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6fbc1-e517-4023-9408-493b8d01e415 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:57.356283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:57.392991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca4457-7e20-4377-a343-9be350d60f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:57.393674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca4457-7e20-4377-a343-9be350d60f62 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:57.393920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca4457-7e20-4377-a343-9be350d60f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:57.394193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca4457-7e20-4377-a343-9be350d60f62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:57.395205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca4457-7e20-4377-a343-9be350d60f62 None admin] RBAC: Adding query filter params (role.id=085ea69bad9746939d7f6aad1904b3f3, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=75f11eaa4b504d0aa4018fb4ffbbd3a2) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:57.395519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca4457-7e20-4377-a343-9be350d60f62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:57.397177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca4457-7e20-4377-a343-9be350d60f62 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:57.412344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 494/977] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:21:57 2022] GET /identity/v3/role_assignments?user.id=75f11eaa4b504d0aa4018fb4ffbbd3a2&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=085ea69bad9746939d7f6aad1904b3f3 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:57.479122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb606703-6070-42aa-ab9a-2e33d518ba72 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:57.519849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99cffe-a16a-4ef9-8ac9-df913e21f45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:57.521058 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99cffe-a16a-4ef9-8ac9-df913e21f45e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:57.521564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99cffe-a16a-4ef9-8ac9-df913e21f45e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:57.522047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99cffe-a16a-4ef9-8ac9-df913e21f45e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:57.531760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99cffe-a16a-4ef9-8ac9-df913e21f45e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:57.533814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99cffe-a16a-4ef9-8ac9-df913e21f45e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6c99cffe-a16a-4ef9-8ac9-df913e21f45e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:21:57.549985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:57.551063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:57.552040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:57.552040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:57.552040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:21:57.552040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:21:57.552040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:21:57.552040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:21:57.552040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:21:57.552040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:21:57.552040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 17:21:57.552040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:21:57.555635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 484/978] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:21:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:57.562441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca4457-7e20-4377-a343-9be350d60f62 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:57.666468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:57.673371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca4457-7e20-4377-a343-9be350d60f62 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:57.708956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede5d69-b172-4b4e-bf52-fdae4343e981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:57.710058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede5d69-b172-4b4e-bf52-fdae4343e981 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:57.710711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede5d69-b172-4b4e-bf52-fdae4343e981 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:57.711192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede5d69-b172-4b4e-bf52-fdae4343e981 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:57.712195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede5d69-b172-4b4e-bf52-fdae4343e981 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:57.712736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede5d69-b172-4b4e-bf52-fdae4343e981 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:57.714375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede5d69-b172-4b4e-bf52-fdae4343e981 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:57.750577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 495/979] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Jul 21 17:21:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:58.021817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535addae-525c-467e-8853-dedf25897a61 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:58.022145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535addae-525c-467e-8853-dedf25897a61 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:58.022370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535addae-525c-467e-8853-dedf25897a61 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:58.027226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 485/980] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:58.032552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee752d-e873-40c8-9a45-827d3123d36d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:58.032845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee752d-e873-40c8-9a45-827d3123d36d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:58.033119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee752d-e873-40c8-9a45-827d3123d36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:58.050694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2fee752d-e873-40c8-9a45-827d3123d36d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:58.051870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2fee752d-e873-40c8-9a45-827d3123d36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:58.121183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 496/981] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:58.144220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535addae-525c-467e-8853-dedf25897a61 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:58.213887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:58.250751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535addae-525c-467e-8853-dedf25897a61 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:58.283801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad54920-ed70-4561-af28-f84bcf72f9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:58.284481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad54920-ed70-4561-af28-f84bcf72f9b3 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:58.284816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad54920-ed70-4561-af28-f84bcf72f9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:58.285088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad54920-ed70-4561-af28-f84bcf72f9b3 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:58.285863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad54920-ed70-4561-af28-f84bcf72f9b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:58.287217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad54920-ed70-4561-af28-f84bcf72f9b3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:58.308337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 486/982] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:21:58 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:21:58.983404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d018-132d-4a8f-9de0-e1baceb0876e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:58.983906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d018-132d-4a8f-9de0-e1baceb0876e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:58.984084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d018-132d-4a8f-9de0-e1baceb0876e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:58.985481 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 497/983] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:21:58.990611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb5f82-7ddb-40a8-a907-c11a32fd433f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:58.990983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb5f82-7ddb-40a8-a907-c11a32fd433f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:58.991172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb5f82-7ddb-40a8-a907-c11a32fd433f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:59.008934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1feb5f82-7ddb-40a8-a907-c11a32fd433f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:59.010448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1feb5f82-7ddb-40a8-a907-c11a32fd433f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:59.087218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 487/984] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:21:59.110164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642d018-132d-4a8f-9de0-e1baceb0876e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:59.196049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:59.214321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642d018-132d-4a8f-9de0-e1baceb0876e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:59.248229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d92eb-fec8-41fb-bc1c-dcf54f446754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:59.249065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d92eb-fec8-41fb-bc1c-dcf54f446754 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:59.249348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d92eb-fec8-41fb-bc1c-dcf54f446754 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:59.249623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d92eb-fec8-41fb-bc1c-dcf54f446754 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:59.250272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d92eb-fec8-41fb-bc1c-dcf54f446754 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:59.251642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d92eb-fec8-41fb-bc1c-dcf54f446754 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-108d92eb-fec8-41fb-bc1c-dcf54f446754 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:59.264159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:59.265195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:21:59.266034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:59.267335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 498/985] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:21:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:59.271691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb5f82-7ddb-40a8-a907-c11a32fd433f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:59.330323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:59.376857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb5f82-7ddb-40a8-a907-c11a32fd433f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:59.412584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73472a29-9f20-42af-b10a-0df404633958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:59.413257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73472a29-9f20-42af-b10a-0df404633958 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:59.413630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73472a29-9f20-42af-b10a-0df404633958 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:59.414132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73472a29-9f20-42af-b10a-0df404633958 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:59.414947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73472a29-9f20-42af-b10a-0df404633958 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:59.415255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73472a29-9f20-42af-b10a-0df404633958 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:59.416654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73472a29-9f20-42af-b10a-0df404633958 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:59.421810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 488/986] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jul 21 17:21:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:59.427444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d92eb-fec8-41fb-bc1c-dcf54f446754 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:59.481670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a2ace-b4a2-4c0c-a577-7b70d2abfc1c None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:59.481853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a2ace-b4a2-4c0c-a577-7b70d2abfc1c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:59.482136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a2ace-b4a2-4c0c-a577-7b70d2abfc1c None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:59.483516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 489/987] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:21:59.488289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edefe37-5077-4f1b-9e55-5d66f8fcb05f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:59.488602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edefe37-5077-4f1b-9e55-5d66f8fcb05f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:59.488877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edefe37-5077-4f1b-9e55-5d66f8fcb05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:59.508369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8edefe37-5077-4f1b-9e55-5d66f8fcb05f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:21:59.509577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8edefe37-5077-4f1b-9e55-5d66f8fcb05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:21:59.539353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d92eb-fec8-41fb-bc1c-dcf54f446754 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:59.576078 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575e5c-fa35-423f-8b98-74b32305b41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:59.576819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575e5c-fa35-423f-8b98-74b32305b41e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:59.577050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575e5c-fa35-423f-8b98-74b32305b41e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:59.577323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575e5c-fa35-423f-8b98-74b32305b41e None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:59.578109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22575e5c-fa35-423f-8b98-74b32305b41e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:59.578419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22575e5c-fa35-423f-8b98-74b32305b41e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:59.578896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 490/988] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:21:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:21:59.579849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22575e5c-fa35-423f-8b98-74b32305b41e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:59.589858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 499/989] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jul 21 17:21:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:59.590922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edefe37-5077-4f1b-9e55-5d66f8fcb05f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:59.602039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575e5c-fa35-423f-8b98-74b32305b41e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:59.619244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:59.644353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:59.701172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edefe37-5077-4f1b-9e55-5d66f8fcb05f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:59.710931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575e5c-fa35-423f-8b98-74b32305b41e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:59.736478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00e7d0-0c7d-4e3a-a72b-f1fd7f19824b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:59.737142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00e7d0-0c7d-4e3a-a72b-f1fd7f19824b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:59.737357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00e7d0-0c7d-4e3a-a72b-f1fd7f19824b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:59.737634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00e7d0-0c7d-4e3a-a72b-f1fd7f19824b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:59.738459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00e7d0-0c7d-4e3a-a72b-f1fd7f19824b None admin] RBAC: Adding query filter params (interface=admin, service_id=c86170e37a6a4a089e80f9ab3c7f77a2, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:59.738738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00e7d0-0c7d-4e3a-a72b-f1fd7f19824b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:59.740100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00e7d0-0c7d-4e3a-a72b-f1fd7f19824b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:59.746019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 491/990] 10.222.0.57 () {56 vars in 1249 bytes} [Thu Jul 21 17:21:59 2022] GET /identity/v3/endpoints?service_id=c86170e37a6a4a089e80f9ab3c7f77a2&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:21:59.749149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fb74a-b3f1-4276-b283-958e282f76b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:59.750026 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fb74a-b3f1-4276-b283-958e282f76b6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:59.750578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fb74a-b3f1-4276-b283-958e282f76b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:59.750859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fb74a-b3f1-4276-b283-958e282f76b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:59.751131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00e7d0-0c7d-4e3a-a72b-f1fd7f19824b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:59.751514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fb74a-b3f1-4276-b283-958e282f76b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:59.752963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fb74a-b3f1-4276-b283-958e282f76b6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-a94fb74a-b3f1-4276-b283-958e282f76b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:21:59.767799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:21:59.769146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:21:59.770138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 500/991] 10.222.0.57 () {56 vars in 1105 bytes} [Thu Jul 21 17:21:59 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:21:59.774704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fb74a-b3f1-4276-b283-958e282f76b6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:21:59.855555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:59.857570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00e7d0-0c7d-4e3a-a72b-f1fd7f19824b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:59.871480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:21:59.884455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fb74a-b3f1-4276-b283-958e282f76b6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:21:59.893681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6675a-5dcc-4b5f-938b-318f3ae6aec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:59.894348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6675a-5dcc-4b5f-938b-318f3ae6aec9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:59.894625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6675a-5dcc-4b5f-938b-318f3ae6aec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:59.894895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6675a-5dcc-4b5f-938b-318f3ae6aec9 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:59.895609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6675a-5dcc-4b5f-938b-318f3ae6aec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:59.896983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6675a-5dcc-4b5f-938b-318f3ae6aec9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:59.903995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 492/992] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jul 21 17:21:59 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:21:59.924830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc144747-8860-4a35-b6bd-1c6b75fa1bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:21:59.925525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc144747-8860-4a35-b6bd-1c6b75fa1bdb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:21:59.925809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc144747-8860-4a35-b6bd-1c6b75fa1bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:21:59.926084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc144747-8860-4a35-b6bd-1c6b75fa1bdb None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:21:59.926815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc144747-8860-4a35-b6bd-1c6b75fa1bdb None admin] RBAC: Adding query filter params (name=image) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:21:59.927123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc144747-8860-4a35-b6bd-1c6b75fa1bdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:21:59.928587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc144747-8860-4a35-b6bd-1c6b75fa1bdb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:21:59.942393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 501/993] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:21:59.943418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6675a-5dcc-4b5f-938b-318f3ae6aec9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:00.053026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6675a-5dcc-4b5f-938b-318f3ae6aec9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:00.090167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976782e-4349-4c34-ab09-6c250f927874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:00.090841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976782e-4349-4c34-ab09-6c250f927874 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:00.091123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976782e-4349-4c34-ab09-6c250f927874 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:00.091397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976782e-4349-4c34-ab09-6c250f927874 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:00.092106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976782e-4349-4c34-ab09-6c250f927874 None admin] RBAC: Adding query filter params (type=image) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:00.092516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976782e-4349-4c34-ab09-6c250f927874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:00.093877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976782e-4349-4c34-ab09-6c250f927874 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:00.099040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 493/994] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:21:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:00.398271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01572b99-a47c-4f54-8186-ccad6bd70b48 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:00.398786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01572b99-a47c-4f54-8186-ccad6bd70b48 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:00.399088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01572b99-a47c-4f54-8186-ccad6bd70b48 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:00.400612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 502/995] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:00.405709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e09613-078b-4bc5-a3b5-9e3636e5626d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:00.406068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e09613-078b-4bc5-a3b5-9e3636e5626d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:00.406438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e09613-078b-4bc5-a3b5-9e3636e5626d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:00.423565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c0e09613-078b-4bc5-a3b5-9e3636e5626d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:00.424795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c0e09613-078b-4bc5-a3b5-9e3636e5626d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:00.460050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:00.495268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 494/996] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:00.517364 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01572b99-a47c-4f54-8186-ccad6bd70b48 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:00.622342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01572b99-a47c-4f54-8186-ccad6bd70b48 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:00.656939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29e349-53a5-470c-8683-32fec77561dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:00.657655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29e349-53a5-470c-8683-32fec77561dd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:00.657939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29e349-53a5-470c-8683-32fec77561dd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:00.658218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29e349-53a5-470c-8683-32fec77561dd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:00.658865 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29e349-53a5-470c-8683-32fec77561dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:00.660226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29e349-53a5-470c-8683-32fec77561dd None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-0f29e349-53a5-470c-8683-32fec77561dd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:00.672009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:00.673025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:00.673880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 503/997] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:22:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:22:00.681528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e09613-078b-4bc5-a3b5-9e3636e5626d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:00.781922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e09613-078b-4bc5-a3b5-9e3636e5626d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:00.816372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f53b5-91d7-41bf-9268-9d9bfadc34c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:00.817040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f53b5-91d7-41bf-9268-9d9bfadc34c1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:00.817323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f53b5-91d7-41bf-9268-9d9bfadc34c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:00.817599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f53b5-91d7-41bf-9268-9d9bfadc34c1 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:00.818300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f53b5-91d7-41bf-9268-9d9bfadc34c1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:00.818606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f53b5-91d7-41bf-9268-9d9bfadc34c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:00.820705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f53b5-91d7-41bf-9268-9d9bfadc34c1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:00.830496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 495/998] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:22:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:00.831607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29e349-53a5-470c-8683-32fec77561dd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:00.836064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:00.939018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29e349-53a5-470c-8683-32fec77561dd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:00.973980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7fd57-7814-46ed-9d3e-366b33b6ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:00.974942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7fd57-7814-46ed-9d3e-366b33b6ac27 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:00.975451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7fd57-7814-46ed-9d3e-366b33b6ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:00.975992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7fd57-7814-46ed-9d3e-366b33b6ac27 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:00.977104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7fd57-7814-46ed-9d3e-366b33b6ac27 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:00.977631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7fd57-7814-46ed-9d3e-366b33b6ac27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:00.979191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7fd57-7814-46ed-9d3e-366b33b6ac27 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:00.984534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 504/999] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:22:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:00.990143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f53b5-91d7-41bf-9268-9d9bfadc34c1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:01.025963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:01.097264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f53b5-91d7-41bf-9268-9d9bfadc34c1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:01.131856 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241edcc-1bb0-4e26-b045-838c68e2999f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:01.133093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241edcc-1bb0-4e26-b045-838c68e2999f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:01.133370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241edcc-1bb0-4e26-b045-838c68e2999f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:01.133635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241edcc-1bb0-4e26-b045-838c68e2999f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:01.134417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241edcc-1bb0-4e26-b045-838c68e2999f None admin] RBAC: Adding query filter params (interface=public, service_id=18268138c0394c0696e706f21dd7f648, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:01.134725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241edcc-1bb0-4e26-b045-838c68e2999f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:01.136073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241edcc-1bb0-4e26-b045-838c68e2999f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:01.141280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 496/1000] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jul 21 17:22:00 2022] GET /identity/v3/endpoints?service_id=18268138c0394c0696e706f21dd7f648&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:01.146595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7fd57-7814-46ed-9d3e-366b33b6ac27 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:01.216023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:01.252919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7fd57-7814-46ed-9d3e-366b33b6ac27 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:01.286613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c3d23-aa2f-4361-9c2e-6dfd17973e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:01.287275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c3d23-aa2f-4361-9c2e-6dfd17973e9e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:01.287557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c3d23-aa2f-4361-9c2e-6dfd17973e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:01.287831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c3d23-aa2f-4361-9c2e-6dfd17973e9e None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:01.288567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c3d23-aa2f-4361-9c2e-6dfd17973e9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:01.290088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c3d23-aa2f-4361-9c2e-6dfd17973e9e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:01.296427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 505/1001] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jul 21 17:22:01 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:22:01.974748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3da68-577e-4248-8004-bd3753b5d285 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:01.975327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3da68-577e-4248-8004-bd3753b5d285 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:01.975628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3da68-577e-4248-8004-bd3753b5d285 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:01.977402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 497/1002] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:01.983001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339250-971b-4e74-94eb-b9b44c2092e4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:01.983481 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339250-971b-4e74-94eb-b9b44c2092e4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:01.983567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339250-971b-4e74-94eb-b9b44c2092e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:02.004265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6b339250-971b-4e74-94eb-b9b44c2092e4 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:02.005839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6b339250-971b-4e74-94eb-b9b44c2092e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:02.084688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 506/1003] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:02.102748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3da68-577e-4248-8004-bd3753b5d285 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:02.173364 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d2a35-56d4-4db8-8c9c-1861bff05359 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:02.173696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d2a35-56d4-4db8-8c9c-1861bff05359 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:02.173931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d2a35-56d4-4db8-8c9c-1861bff05359 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:02.175554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 507/1004] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:02.180935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799aeb9d-834f-4813-889d-34367dc15d25 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:02.181227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799aeb9d-834f-4813-889d-34367dc15d25 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:02.181512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799aeb9d-834f-4813-889d-34367dc15d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:02.199321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-799aeb9d-834f-4813-889d-34367dc15d25 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:02.200766 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-799aeb9d-834f-4813-889d-34367dc15d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:02.217865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:02.226834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3da68-577e-4248-8004-bd3753b5d285 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:02.261843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:02.267360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380382c5-41db-4534-98ea-cae1403f0fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:02.268251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380382c5-41db-4534-98ea-cae1403f0fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:02.268633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380382c5-41db-4534-98ea-cae1403f0fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:02.268981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380382c5-41db-4534-98ea-cae1403f0fc6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:02.269704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380382c5-41db-4534-98ea-cae1403f0fc6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:02.271292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380382c5-41db-4534-98ea-cae1403f0fc6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-380382c5-41db-4534-98ea-cae1403f0fc6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:02.287000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:02.288151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:02.289649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 498/1005] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Jul 21 17:22:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:02.295096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380382c5-41db-4534-98ea-cae1403f0fc6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:02.305042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 508/1006] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:22:02.319770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799aeb9d-834f-4813-889d-34367dc15d25 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:02.418011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380382c5-41db-4534-98ea-cae1403f0fc6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:02.443682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799aeb9d-834f-4813-889d-34367dc15d25 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:02.457374 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c07a39-c0b7-4ed7-abf6-2cd41a1ef63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:02.457972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c07a39-c0b7-4ed7-abf6-2cd41a1ef63f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:02.458371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c07a39-c0b7-4ed7-abf6-2cd41a1ef63f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:02.458653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c07a39-c0b7-4ed7-abf6-2cd41a1ef63f None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:02.459478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c07a39-c0b7-4ed7-abf6-2cd41a1ef63f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:02.459783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c07a39-c0b7-4ed7-abf6-2cd41a1ef63f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:02.461213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c07a39-c0b7-4ed7-abf6-2cd41a1ef63f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:02.471174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 499/1007] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jul 21 17:22:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:02.472110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c07a39-c0b7-4ed7-abf6-2cd41a1ef63f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:02.483868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142f429-9560-463c-9c47-1af045cf396b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:02.484931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142f429-9560-463c-9c47-1af045cf396b None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:02.485215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142f429-9560-463c-9c47-1af045cf396b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:02.485490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142f429-9560-463c-9c47-1af045cf396b None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:02.486526 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142f429-9560-463c-9c47-1af045cf396b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:02.488038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142f429-9560-463c-9c47-1af045cf396b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:02.502631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:02.503551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 509/1008] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:22:02 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:22:02.589640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c07a39-c0b7-4ed7-abf6-2cd41a1ef63f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:02.634060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf18380-d92b-42c2-8f87-cb108e85e212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:02.634748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18380-d92b-42c2-8f87-cb108e85e212 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:02.635018 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18380-d92b-42c2-8f87-cb108e85e212 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:02.635289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18380-d92b-42c2-8f87-cb108e85e212 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:02.635993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf18380-d92b-42c2-8f87-cb108e85e212 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:02.636326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf18380-d92b-42c2-8f87-cb108e85e212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:02.637700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf18380-d92b-42c2-8f87-cb108e85e212 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:02.648560 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 500/1009] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Jul 21 17:22:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:02.650487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142f429-9560-463c-9c47-1af045cf396b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:02.707131 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:02.756177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142f429-9560-463c-9c47-1af045cf396b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:02.790398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ee1f2-269b-468f-8b8a-f11aea58f000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:02.791061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ee1f2-269b-468f-8b8a-f11aea58f000 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:02.791354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ee1f2-269b-468f-8b8a-f11aea58f000 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:02.791621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ee1f2-269b-468f-8b8a-f11aea58f000 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:02.792382 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ee1f2-269b-468f-8b8a-f11aea58f000 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:02.793755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ee1f2-269b-468f-8b8a-f11aea58f000 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:02.816332 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 510/1010] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jul 21 17:22:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:22:03.340767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e26865-61c6-4017-8bf5-22d96117a721 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:03.341314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e26865-61c6-4017-8bf5-22d96117a721 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:03.341314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e26865-61c6-4017-8bf5-22d96117a721 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:03.342815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 501/1011] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:03.347747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48c12-d66e-4e66-baee-6e263d90c0b2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:03.348157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48c12-d66e-4e66-baee-6e263d90c0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:03.348373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48c12-d66e-4e66-baee-6e263d90c0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:03.365061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-67d48c12-d66e-4e66-baee-6e263d90c0b2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:03.366219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-67d48c12-d66e-4e66-baee-6e263d90c0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:03.438715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 511/1012] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:03 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:03.461388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e26865-61c6-4017-8bf5-22d96117a721 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:03.547176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:03.565131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e26865-61c6-4017-8bf5-22d96117a721 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:03.599103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c6e51-51a2-43ab-8502-82e91f3a7f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:03.599829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c6e51-51a2-43ab-8502-82e91f3a7f98 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:03.600107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c6e51-51a2-43ab-8502-82e91f3a7f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:03.600403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c6e51-51a2-43ab-8502-82e91f3a7f98 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:03.601320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c6e51-51a2-43ab-8502-82e91f3a7f98 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:03.602493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c6e51-51a2-43ab-8502-82e91f3a7f98 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-af0c6e51-51a2-43ab-8502-82e91f3a7f98 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:03.614240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:03.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 17:22:03.616155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:03.620640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 502/1013] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:22:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:03.622138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d48c12-d66e-4e66-baee-6e263d90c0b2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:03.660650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:03.726443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d48c12-d66e-4e66-baee-6e263d90c0b2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:03.759959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a4e7d-2b1a-4105-9a62-753b1f90ff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:03.760660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a4e7d-2b1a-4105-9a62-753b1f90ff28 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:03.760944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a4e7d-2b1a-4105-9a62-753b1f90ff28 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:03.761220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a4e7d-2b1a-4105-9a62-753b1f90ff28 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:03.761919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a4e7d-2b1a-4105-9a62-753b1f90ff28 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:03.762221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a4e7d-2b1a-4105-9a62-753b1f90ff28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:03.763588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a4e7d-2b1a-4105-9a62-753b1f90ff28 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:03.768767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 512/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:22:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:03.774454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c6e51-51a2-43ab-8502-82e91f3a7f98 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:03.875697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c6e51-51a2-43ab-8502-82e91f3a7f98 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:03.909302 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73214e5-a72d-4303-b09f-61ec384e854e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:03.909962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73214e5-a72d-4303-b09f-61ec384e854e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:03.910240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73214e5-a72d-4303-b09f-61ec384e854e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:03.910514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73214e5-a72d-4303-b09f-61ec384e854e None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:03.911215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73214e5-a72d-4303-b09f-61ec384e854e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:03.911519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73214e5-a72d-4303-b09f-61ec384e854e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:03.916193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73214e5-a72d-4303-b09f-61ec384e854e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:03.922353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 503/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Jul 21 17:22:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:03.927930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a4e7d-2b1a-4105-9a62-753b1f90ff28 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:04.028887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a4e7d-2b1a-4105-9a62-753b1f90ff28 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:04.030434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:04.064570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da8dde-1f33-41cd-afe8-33b8925e3b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:04.065224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da8dde-1f33-41cd-afe8-33b8925e3b77 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:04.065506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da8dde-1f33-41cd-afe8-33b8925e3b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:04.065779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da8dde-1f33-41cd-afe8-33b8925e3b77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:04.066515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da8dde-1f33-41cd-afe8-33b8925e3b77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:04.067883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da8dde-1f33-41cd-afe8-33b8925e3b77 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:04.087612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 513/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jul 21 17:22:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:22:04.622457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe33c-05be-4a73-ac4e-2370a007ab50 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:04.623013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe33c-05be-4a73-ac4e-2370a007ab50 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:04.623013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe33c-05be-4a73-ac4e-2370a007ab50 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:04.624674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 504/1017] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:04.630182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efe332-c914-4688-bfe6-165f724a5dfb None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:04.630624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efe332-c914-4688-bfe6-165f724a5dfb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:04.630811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efe332-c914-4688-bfe6-165f724a5dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:04.648707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-11efe332-c914-4688-bfe6-165f724a5dfb None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:04.649899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-11efe332-c914-4688-bfe6-165f724a5dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:04.727347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 514/1018] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:04.750315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafe33c-05be-4a73-ac4e-2370a007ab50 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:04.767647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:04.865168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafe33c-05be-4a73-ac4e-2370a007ab50 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:04.867500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6c08f-de35-463d-a5ae-c55f542863c3 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:04.867785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6c08f-de35-463d-a5ae-c55f542863c3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:04.868071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6c08f-de35-463d-a5ae-c55f542863c3 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:04.869477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 515/1019] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:22:04.874571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b5041-b3b7-465e-a021-c0aae13d8fe7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:04.874806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b5041-b3b7-465e-a021-c0aae13d8fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:04.875069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b5041-b3b7-465e-a021-c0aae13d8fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:04.891841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-242b5041-b3b7-465e-a021-c0aae13d8fe7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:04.893061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-242b5041-b3b7-465e-a021-c0aae13d8fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:04.902625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157cde1-5c33-46a0-bc65-ae940efbf943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:04.903419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157cde1-5c33-46a0-bc65-ae940efbf943 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:04.903699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157cde1-5c33-46a0-bc65-ae940efbf943 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:04.903971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157cde1-5c33-46a0-bc65-ae940efbf943 None admin] PATH_INFO: `/v3/services/image` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:04.904669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157cde1-5c33-46a0-bc65-ae940efbf943 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:04.906019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157cde1-5c33-46a0-bc65-ae940efbf943 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-7157cde1-5c33-46a0-bc65-ae940efbf943 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:04.918958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:04.919992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:04.920877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 505/1020] 10.222.0.57 () {56 vars in 1105 bytes} [Thu Jul 21 17:22:04 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:04.926179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157cde1-5c33-46a0-bc65-ae940efbf943 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:04.974693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 516/1021] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:22:04.997854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b5041-b3b7-465e-a021-c0aae13d8fe7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:05.006303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:05.013477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:05.031459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157cde1-5c33-46a0-bc65-ae940efbf943 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:05.071866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ad702-db19-47ea-8dde-e42ddc6a1a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:05.072568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ad702-db19-47ea-8dde-e42ddc6a1a8c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:05.072858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ad702-db19-47ea-8dde-e42ddc6a1a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:05.073400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ad702-db19-47ea-8dde-e42ddc6a1a8c None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:05.074095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ad702-db19-47ea-8dde-e42ddc6a1a8c None admin] RBAC: Adding query filter params (name=image) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:05.074405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ad702-db19-47ea-8dde-e42ddc6a1a8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:05.075770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ad702-db19-47ea-8dde-e42ddc6a1a8c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:05.080799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 506/1022] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:05.089326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ad702-db19-47ea-8dde-e42ddc6a1a8c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:05.109875 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b5041-b3b7-465e-a021-c0aae13d8fe7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:05.148372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99ca66-a9af-40b9-a140-942b95cd0ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:05.149179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99ca66-a9af-40b9-a140-942b95cd0ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:05.149461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99ca66-a9af-40b9-a140-942b95cd0ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:05.149738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99ca66-a9af-40b9-a140-942b95cd0ec1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:05.150901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99ca66-a9af-40b9-a140-942b95cd0ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:05.152349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99ca66-a9af-40b9-a140-942b95cd0ec1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:05.154036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 517/1023] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:05.159491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99ca66-a9af-40b9-a140-942b95cd0ec1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:05.204273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ad702-db19-47ea-8dde-e42ddc6a1a8c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:05.240408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c176985-d977-4b13-8213-95c05bb5fc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:05.241084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c176985-d977-4b13-8213-95c05bb5fc51 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:05.242267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c176985-d977-4b13-8213-95c05bb5fc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:05.242549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c176985-d977-4b13-8213-95c05bb5fc51 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:05.243253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c176985-d977-4b13-8213-95c05bb5fc51 None admin] RBAC: Adding query filter params (type=image) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:05.243559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c176985-d977-4b13-8213-95c05bb5fc51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:05.244955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c176985-d977-4b13-8213-95c05bb5fc51 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:05.245465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:05.250484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 507/1024] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:05.256001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c176985-d977-4b13-8213-95c05bb5fc51 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:05.264998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:05.269942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99ca66-a9af-40b9-a140-942b95cd0ec1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:05.313607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9bde8-909c-4941-b2dc-61c8b946bb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:05.314265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9bde8-909c-4941-b2dc-61c8b946bb30 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:05.314544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9bde8-909c-4941-b2dc-61c8b946bb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:05.314819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9bde8-909c-4941-b2dc-61c8b946bb30 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:05.315513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9bde8-909c-4941-b2dc-61c8b946bb30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:05.316972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9bde8-909c-4941-b2dc-61c8b946bb30 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:05.343623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 518/1025] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:22:05 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:22:05.369608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c176985-d977-4b13-8213-95c05bb5fc51 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:05.408721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13c18d-0786-4479-ac19-9c877a3c9407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:05.409770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13c18d-0786-4479-ac19-9c877a3c9407 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:05.410271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13c18d-0786-4479-ac19-9c877a3c9407 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:05.410771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13c18d-0786-4479-ac19-9c877a3c9407 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:05.411942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13c18d-0786-4479-ac19-9c877a3c9407 None admin] RBAC: Adding query filter params (interface=public, service_id=a6f25e62faf84f18a89b7c77d9b60085, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:05.412637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13c18d-0786-4479-ac19-9c877a3c9407 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:05.415967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13c18d-0786-4479-ac19-9c877a3c9407 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:05.425853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 508/1026] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jul 21 17:22:05 2022] GET /identity/v3/endpoints?service_id=a6f25e62faf84f18a89b7c77d9b60085&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:05.432286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9bde8-909c-4941-b2dc-61c8b946bb30 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:05.545271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9bde8-909c-4941-b2dc-61c8b946bb30 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:05.580758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246dfa4-a809-4827-a82c-6849a0269c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:05.581780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246dfa4-a809-4827-a82c-6849a0269c18 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:05.582279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246dfa4-a809-4827-a82c-6849a0269c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:05.582784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246dfa4-a809-4827-a82c-6849a0269c18 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:05.583677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246dfa4-a809-4827-a82c-6849a0269c18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:05.585319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246dfa4-a809-4827-a82c-6849a0269c18 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:05.586717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:05.594300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 519/1027] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jul 21 17:22:05 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:22:06.244935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773dd46-9565-4749-b545-121ed8d15b83 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:06.245426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773dd46-9565-4749-b545-121ed8d15b83 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:06.245426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773dd46-9565-4749-b545-121ed8d15b83 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:06.246930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 509/1028] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:06.251993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d1c23-2657-4b7b-9ac0-1bdc33302989 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:06.252416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d1c23-2657-4b7b-9ac0-1bdc33302989 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:06.252610 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d1c23-2657-4b7b-9ac0-1bdc33302989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:06.269525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-270d1c23-2657-4b7b-9ac0-1bdc33302989 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:06.270710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-270d1c23-2657-4b7b-9ac0-1bdc33302989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:06.344017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 520/1029] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:06.368717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773dd46-9565-4749-b545-121ed8d15b83 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:06.440010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:06.474539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773dd46-9565-4749-b545-121ed8d15b83 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:06.511155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0c6ac-4029-4e82-ab97-16f75452cc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:06.511904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0c6ac-4029-4e82-ab97-16f75452cc81 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:06.512184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0c6ac-4029-4e82-ab97-16f75452cc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:06.512489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0c6ac-4029-4e82-ab97-16f75452cc81 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:06.524441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0c6ac-4029-4e82-ab97-16f75452cc81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-21a0c6ac-4029-4e82-ab97-16f75452cc81 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:06.532474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:06.533412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:22:06.534276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:06.536612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 510/1030] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jul 21 17:22:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:06.540868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d1c23-2657-4b7b-9ac0-1bdc33302989 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:06.643519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d1c23-2657-4b7b-9ac0-1bdc33302989 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:06.671865 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:06.680555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75f8e5-abd2-414b-b8c1-e6b9d4f35f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:06.681224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75f8e5-abd2-414b-b8c1-e6b9d4f35f37 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:06.681502 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75f8e5-abd2-414b-b8c1-e6b9d4f35f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:06.681771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75f8e5-abd2-414b-b8c1-e6b9d4f35f37 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:06.682506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75f8e5-abd2-414b-b8c1-e6b9d4f35f37 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:06.682808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75f8e5-abd2-414b-b8c1-e6b9d4f35f37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:06.684174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75f8e5-abd2-414b-b8c1-e6b9d4f35f37 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:06.697591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 521/1031] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jul 21 17:22:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:06.699395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0c6ac-4029-4e82-ab97-16f75452cc81 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:06.799431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0c6ac-4029-4e82-ab97-16f75452cc81 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:06.827577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:06.834224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd46e7-23a3-44da-927f-202f16bffea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:06.835728 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd46e7-23a3-44da-927f-202f16bffea3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:06.836016 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd46e7-23a3-44da-927f-202f16bffea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:06.836303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd46e7-23a3-44da-927f-202f16bffea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:06.846534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd46e7-23a3-44da-927f-202f16bffea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:06.847971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd46e7-23a3-44da-927f-202f16bffea3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-68cd46e7-23a3-44da-927f-202f16bffea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:22:06.866036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:06.867194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:06.868090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:22:06.868090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:22:06.868090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:06.868090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:06.868090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 511/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:06.874181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75f8e5-abd2-414b-b8c1-e6b9d4f35f37 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:06.987561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75f8e5-abd2-414b-b8c1-e6b9d4f35f37 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:07.020685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26ee3f-dbde-44dc-9d08-1b6d87768406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:07.021345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26ee3f-dbde-44dc-9d08-1b6d87768406 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.021626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26ee3f-dbde-44dc-9d08-1b6d87768406 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.021900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26ee3f-dbde-44dc-9d08-1b6d87768406 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.023192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26ee3f-dbde-44dc-9d08-1b6d87768406 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:07.023546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26ee3f-dbde-44dc-9d08-1b6d87768406 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:07.024948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26ee3f-dbde-44dc-9d08-1b6d87768406 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:07.040379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 522/1033] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:07.042673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd46e7-23a3-44da-927f-202f16bffea3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:07.144768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd46e7-23a3-44da-927f-202f16bffea3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:07.177581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc687e4d-dd15-4d09-8629-e0a76c460e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:07.178375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc687e4d-dd15-4d09-8629-e0a76c460e81 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.179168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc687e4d-dd15-4d09-8629-e0a76c460e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.179445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc687e4d-dd15-4d09-8629-e0a76c460e81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.186752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc687e4d-dd15-4d09-8629-e0a76c460e81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:07.188178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc687e4d-dd15-4d09-8629-e0a76c460e81 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-fc687e4d-dd15-4d09-8629-e0a76c460e81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:22:07.203245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:07.204265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:07.210173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 512/1034] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jul 21 17:22:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:07.210808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26ee3f-dbde-44dc-9d08-1b6d87768406 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:07.261263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:07.311598 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26ee3f-dbde-44dc-9d08-1b6d87768406 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:07.345893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb6f74-6aac-4d4a-beb5-8b76609bee36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:07.346563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb6f74-6aac-4d4a-beb5-8b76609bee36 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.346848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb6f74-6aac-4d4a-beb5-8b76609bee36 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.347121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb6f74-6aac-4d4a-beb5-8b76609bee36 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.347950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb6f74-6aac-4d4a-beb5-8b76609bee36 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:07.348323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb6f74-6aac-4d4a-beb5-8b76609bee36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:07.349742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb6f74-6aac-4d4a-beb5-8b76609bee36 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:07.372195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 523/1035] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jul 21 17:22:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:07.373346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc687e4d-dd15-4d09-8629-e0a76c460e81 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:07.418413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:07.422662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d407b6-765c-4d5f-b9f6-8ba5ab69948d None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.422949 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d407b6-765c-4d5f-b9f6-8ba5ab69948d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.423227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d407b6-765c-4d5f-b9f6-8ba5ab69948d None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.425180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 524/1036] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:07.429783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415665e7-c99c-437d-b0fe-f7ad7295d846 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.430045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415665e7-c99c-437d-b0fe-f7ad7295d846 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.430321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415665e7-c99c-437d-b0fe-f7ad7295d846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.447129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-415665e7-c99c-437d-b0fe-f7ad7295d846 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:07.448589 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-415665e7-c99c-437d-b0fe-f7ad7295d846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:07.480446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc687e4d-dd15-4d09-8629-e0a76c460e81 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:07.514828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5ee21-0c2e-4ac7-8161-2c894053fb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:07.515653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5ee21-0c2e-4ac7-8161-2c894053fb4e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.516030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5ee21-0c2e-4ac7-8161-2c894053fb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.516436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5ee21-0c2e-4ac7-8161-2c894053fb4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.518017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 525/1037] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:22:07.542059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415665e7-c99c-437d-b0fe-f7ad7295d846 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:07.581805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:07.586885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5ee21-0c2e-4ac7-8161-2c894053fb4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:07.588381 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5ee21-0c2e-4ac7-8161-2c894053fb4e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:07.637596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:07.649799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415665e7-c99c-437d-b0fe-f7ad7295d846 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:07.684136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498dd68-80f8-4038-8a45-d7c1fa488bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:07.684867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498dd68-80f8-4038-8a45-d7c1fa488bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.685143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498dd68-80f8-4038-8a45-d7c1fa488bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.685413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498dd68-80f8-4038-8a45-d7c1fa488bb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.686573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498dd68-80f8-4038-8a45-d7c1fa488bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:07.688000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498dd68-80f8-4038-8a45-d7c1fa488bb5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:07.689625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 526/1038] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:07.690850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d5406-f5bc-4b99-b926-f90b57aab1dc None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.691134 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d5406-f5bc-4b99-b926-f90b57aab1dc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.691410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d5406-f5bc-4b99-b926-f90b57aab1dc None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.692783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 527/1039] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:07.696594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 513/1040] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:22:07.697187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d5406-f5bc-4b99-b926-f90b57aab1dc None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:07.699161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eb25b-f6ac-4728-8acc-91f5db7758d2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.699458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eb25b-f6ac-4728-8acc-91f5db7758d2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.699738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eb25b-f6ac-4728-8acc-91f5db7758d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.715959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3c1eb25b-f6ac-4728-8acc-91f5db7758d2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:07.717156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3c1eb25b-f6ac-4728-8acc-91f5db7758d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:07.782683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 514/1041] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:22:07.783516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1eb25b-f6ac-4728-8acc-91f5db7758d2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:07.806850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d5406-f5bc-4b99-b926-f90b57aab1dc None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:07.815299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:07.844331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029273a-7477-4fa8-bde8-47a9dcda058d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:07.845005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029273a-7477-4fa8-bde8-47a9dcda058d None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.845286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029273a-7477-4fa8-bde8-47a9dcda058d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.845567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029273a-7477-4fa8-bde8-47a9dcda058d None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.846261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029273a-7477-4fa8-bde8-47a9dcda058d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:07.847683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029273a-7477-4fa8-bde8-47a9dcda058d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:07.857986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:07.877067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 528/1042] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:22:07 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:22:07.877067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029273a-7477-4fa8-bde8-47a9dcda058d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:07.890983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1eb25b-f6ac-4728-8acc-91f5db7758d2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:07.935824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdb03b-a0c4-494d-84de-eff1864804c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:07.936697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdb03b-a0c4-494d-84de-eff1864804c1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:07.936969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdb03b-a0c4-494d-84de-eff1864804c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:07.937242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdb03b-a0c4-494d-84de-eff1864804c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:07.950905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdb03b-a0c4-494d-84de-eff1864804c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:07.952428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdb03b-a0c4-494d-84de-eff1864804c1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-23bdb03b-a0c4-494d-84de-eff1864804c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:22:07.971101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:07.972269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:07.973723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:22:07.973723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:22:07.973723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:07.973723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:07.973723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 515/1043] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:07.979110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdb03b-a0c4-494d-84de-eff1864804c1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:08.020905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029273a-7477-4fa8-bde8-47a9dcda058d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:08.059689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098cd01-63cb-4489-abdd-2330906bc344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:08.060929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098cd01-63cb-4489-abdd-2330906bc344 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:08.061436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098cd01-63cb-4489-abdd-2330906bc344 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:08.061851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098cd01-63cb-4489-abdd-2330906bc344 None admin] PATH_INFO: `/v3/services/image` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:08.062518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098cd01-63cb-4489-abdd-2330906bc344 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:08.063908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098cd01-63cb-4489-abdd-2330906bc344 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-a098cd01-63cb-4489-abdd-2330906bc344 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:08.077079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:08.078119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 17:22:08.078985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:08.084479 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 529/1044] 10.222.0.57 () {56 vars in 1105 bytes} [Thu Jul 21 17:22:07 2022] GET /identity/v3/services/image => generated 86 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:08.086684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098cd01-63cb-4489-abdd-2330906bc344 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:08.098692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdb03b-a0c4-494d-84de-eff1864804c1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:08.117714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:08.129432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:08.140337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fd691-214c-4c6f-8d4a-f858266f3678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:08.141252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fd691-214c-4c6f-8d4a-f858266f3678 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:08.141747 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fd691-214c-4c6f-8d4a-f858266f3678 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:08.142164 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fd691-214c-4c6f-8d4a-f858266f3678 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:08.143038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fd691-214c-4c6f-8d4a-f858266f3678 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:08.143492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fd691-214c-4c6f-8d4a-f858266f3678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:08.145083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fd691-214c-4c6f-8d4a-f858266f3678 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:08.157853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 516/1045] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:08.164116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fd691-214c-4c6f-8d4a-f858266f3678 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:08.199848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098cd01-63cb-4489-abdd-2330906bc344 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:08.237702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc0fb8-4f6c-4656-a4cc-39ac17a2a3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:08.238377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc0fb8-4f6c-4656-a4cc-39ac17a2a3ea None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:08.238659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc0fb8-4f6c-4656-a4cc-39ac17a2a3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:08.238933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc0fb8-4f6c-4656-a4cc-39ac17a2a3ea None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:08.239639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc0fb8-4f6c-4656-a4cc-39ac17a2a3ea None admin] RBAC: Adding query filter params (name=image) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:08.239947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc0fb8-4f6c-4656-a4cc-39ac17a2a3ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:08.242349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc0fb8-4f6c-4656-a4cc-39ac17a2a3ea None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:08.253864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 530/1046] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:08.255567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc0fb8-4f6c-4656-a4cc-39ac17a2a3ea None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:08.269179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fd691-214c-4c6f-8d4a-f858266f3678 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:08.305547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaebea-199d-4ca1-bdaf-56279efdec55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:08.306666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaebea-199d-4ca1-bdaf-56279efdec55 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:08.307151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaebea-199d-4ca1-bdaf-56279efdec55 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:08.307617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaebea-199d-4ca1-bdaf-56279efdec55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:08.314603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddaebea-199d-4ca1-bdaf-56279efdec55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:08.316025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddaebea-199d-4ca1-bdaf-56279efdec55 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6ddaebea-199d-4ca1-bdaf-56279efdec55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:22:08.330125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:08.331101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:08.332040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:22:08.332040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:22:08.332040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:22:08.332040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:22:08.332040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:22:08.332040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:22:08.332040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:22:08.332040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:08.332040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 517/1047] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:22:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:22:08.337003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaebea-199d-4ca1-bdaf-56279efdec55 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:08.359444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:08.371162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc0fb8-4f6c-4656-a4cc-39ac17a2a3ea None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:08.389369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:08.407771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75ad8b-ae2f-4f03-970b-e6289547a7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:08.408472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75ad8b-ae2f-4f03-970b-e6289547a7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:08.408756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75ad8b-ae2f-4f03-970b-e6289547a7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:08.409036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75ad8b-ae2f-4f03-970b-e6289547a7e2 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:08.409746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75ad8b-ae2f-4f03-970b-e6289547a7e2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:08.410051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75ad8b-ae2f-4f03-970b-e6289547a7e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:08.411412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75ad8b-ae2f-4f03-970b-e6289547a7e2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:08.416863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 531/1048] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:08.422417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75ad8b-ae2f-4f03-970b-e6289547a7e2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:08.451410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaebea-199d-4ca1-bdaf-56279efdec55 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:08.486583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc1e1b-d502-4557-af91-39cb1b140667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:08.487303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc1e1b-d502-4557-af91-39cb1b140667 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:08.487560 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc1e1b-d502-4557-af91-39cb1b140667 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:08.487830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc1e1b-d502-4557-af91-39cb1b140667 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:08.488647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc1e1b-d502-4557-af91-39cb1b140667 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:08.489096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc1e1b-d502-4557-af91-39cb1b140667 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:08.490361 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc1e1b-d502-4557-af91-39cb1b140667 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:08.503907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 518/1049] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:22:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:08.509859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc1e1b-d502-4557-af91-39cb1b140667 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:08.529892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75ad8b-ae2f-4f03-970b-e6289547a7e2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:08.565994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb90ad5-d86e-49c5-a2e3-3c8aa8309a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:08.566657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb90ad5-d86e-49c5-a2e3-3c8aa8309a8a None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:08.566937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb90ad5-d86e-49c5-a2e3-3c8aa8309a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:08.567233 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb90ad5-d86e-49c5-a2e3-3c8aa8309a8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:08.567937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb90ad5-d86e-49c5-a2e3-3c8aa8309a8a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:08.569513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb90ad5-d86e-49c5-a2e3-3c8aa8309a8a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:08.590924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 532/1050] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jul 21 17:22:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:22:08.623804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc1e1b-d502-4557-af91-39cb1b140667 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:08.626631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:08.671985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8d669-eeee-453b-a146-7b77d7a68e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:08.672687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8d669-eeee-453b-a146-7b77d7a68e60 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:08.672972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8d669-eeee-453b-a146-7b77d7a68e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:08.673245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8d669-eeee-453b-a146-7b77d7a68e60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:08.674087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8d669-eeee-453b-a146-7b77d7a68e60 None admin] RBAC: Adding query filter params (role.id=085ea69bad9746939d7f6aad1904b3f3, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=380d6fe7057d44bfb20b0ae2bb1e664f) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:08.674395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8d669-eeee-453b-a146-7b77d7a68e60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:08.675848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8d669-eeee-453b-a146-7b77d7a68e60 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:08.682886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 519/1051] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:22:08 2022] GET /identity/v3/role_assignments?user.id=380d6fe7057d44bfb20b0ae2bb1e664f&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=085ea69bad9746939d7f6aad1904b3f3 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:10.004872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f18d9-6ecc-42da-9edc-7d0edc758615 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.005358 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f18d9-6ecc-42da-9edc-7d0edc758615 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.005440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f18d9-6ecc-42da-9edc-7d0edc758615 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.007359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 533/1052] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:10.012558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f15593-39a9-4dfb-a900-5d5f82799ccf None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.012972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f15593-39a9-4dfb-a900-5d5f82799ccf None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.013185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f15593-39a9-4dfb-a900-5d5f82799ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.030947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e0f15593-39a9-4dfb-a900-5d5f82799ccf None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:10.032126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e0f15593-39a9-4dfb-a900-5d5f82799ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:10.103276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 520/1053] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:10.126217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f18d9-6ecc-42da-9edc-7d0edc758615 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:10.139278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:10.233916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f18d9-6ecc-42da-9edc-7d0edc758615 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:10.268718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa76b2-8029-411c-b3cd-947e76b8c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:10.269514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa76b2-8029-411c-b3cd-947e76b8c253 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.269796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa76b2-8029-411c-b3cd-947e76b8c253 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.270071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa76b2-8029-411c-b3cd-947e76b8c253 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.281496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa76b2-8029-411c-b3cd-947e76b8c253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-5afa76b2-8029-411c-b3cd-947e76b8c253 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:10.289188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:10.290097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:10.291021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:10.292623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 534/1054] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:22:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:10.296483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f15593-39a9-4dfb-a900-5d5f82799ccf None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:10.393959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:10.401704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f15593-39a9-4dfb-a900-5d5f82799ccf None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:10.436823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd88dea-50c3-46de-bff3-fa33309e7000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:10.437500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd88dea-50c3-46de-bff3-fa33309e7000 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.437783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd88dea-50c3-46de-bff3-fa33309e7000 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.438061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd88dea-50c3-46de-bff3-fa33309e7000 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.438800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd88dea-50c3-46de-bff3-fa33309e7000 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:10.439108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd88dea-50c3-46de-bff3-fa33309e7000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:10.440512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd88dea-50c3-46de-bff3-fa33309e7000 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:10.449249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 521/1055] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:22:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:10.454732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa76b2-8029-411c-b3cd-947e76b8c253 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:10.527500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:10.560341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa76b2-8029-411c-b3cd-947e76b8c253 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:10.594538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca8670-c1de-4eb4-9f81-835e791d6454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:10.595415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca8670-c1de-4eb4-9f81-835e791d6454 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.595746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca8670-c1de-4eb4-9f81-835e791d6454 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.596027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca8670-c1de-4eb4-9f81-835e791d6454 None admin] PATH_INFO: `/v3/users/060fb4a9b1ac480fbc5a10270a2e49e6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.609896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca8670-c1de-4eb4-9f81-835e791d6454 None admin] RBAC: Authorizing `identity:get_user(user_id=060fb4a9b1ac480fbc5a10270a2e49e6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:10.611370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca8670-c1de-4eb4-9f81-835e791d6454 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:10.613293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 535/1056] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:10 2022] GET /identity/v3/users/060fb4a9b1ac480fbc5a10270a2e49e6 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:10.619123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd88dea-50c3-46de-bff3-fa33309e7000 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:10.654256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594fbe-d1d3-4efa-861a-7e84f840e72c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.654783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594fbe-d1d3-4efa-861a-7e84f840e72c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.655070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594fbe-d1d3-4efa-861a-7e84f840e72c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.656595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 536/1057] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:22:10.661577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbe2f8-b857-43e7-9436-c93547c9c4c7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.661873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbe2f8-b857-43e7-9436-c93547c9c4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.662156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbe2f8-b857-43e7-9436-c93547c9c4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.679398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2cdbe2f8-b857-43e7-9436-c93547c9c4c7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:10.680627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2cdbe2f8-b857-43e7-9436-c93547c9c4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:10.731454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd88dea-50c3-46de-bff3-fa33309e7000 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:10.750945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 537/1058] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:22:10.752253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bb8b4-e27f-42c7-b75e-36fb10c30890 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.752584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bb8b4-e27f-42c7-b75e-36fb10c30890 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.752882 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bb8b4-e27f-42c7-b75e-36fb10c30890 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.754295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 538/1059] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:10.760020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10641ca-94b6-41be-9cd8-b597c73bc194 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.760334 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10641ca-94b6-41be-9cd8-b597c73bc194 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.760614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10641ca-94b6-41be-9cd8-b597c73bc194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.768797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a186eca-633a-43b0-96fd-59ee536a81c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:10.769443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a186eca-633a-43b0-96fd-59ee536a81c3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.769777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a186eca-633a-43b0-96fd-59ee536a81c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.770051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a186eca-633a-43b0-96fd-59ee536a81c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.778791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a10641ca-94b6-41be-9cd8-b597c73bc194 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:10.779905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a10641ca-94b6-41be-9cd8-b597c73bc194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:10.794874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:10.834048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:10.839735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a186eca-633a-43b0-96fd-59ee536a81c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:10.841207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a186eca-633a-43b0-96fd-59ee536a81c3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:10.849495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 539/1060] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:22:10.850330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10641ca-94b6-41be-9cd8-b597c73bc194 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:10.940740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 522/1061] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:22:10.941437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a186eca-633a-43b0-96fd-59ee536a81c3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:10.955122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10641ca-94b6-41be-9cd8-b597c73bc194 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:10.988348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba24475-2dcb-4f34-8390-b9c1b85aa043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:10.989027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba24475-2dcb-4f34-8390-b9c1b85aa043 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:10.989310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba24475-2dcb-4f34-8390-b9c1b85aa043 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:10.989584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba24475-2dcb-4f34-8390-b9c1b85aa043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:10.996501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:11.039394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a186eca-633a-43b0-96fd-59ee536a81c3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:11.044664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:11.058170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba24475-2dcb-4f34-8390-b9c1b85aa043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:11.059622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba24475-2dcb-4f34-8390-b9c1b85aa043 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:11.074040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fa720-7f34-4313-be0c-76ba58eab771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:11.074852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa720-7f34-4313-be0c-76ba58eab771 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:11.075135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa720-7f34-4313-be0c-76ba58eab771 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:11.075414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fa720-7f34-4313-be0c-76ba58eab771 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:11.085830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fa720-7f34-4313-be0c-76ba58eab771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-0f7fa720-7f34-4313-be0c-76ba58eab771 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:11.094035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:11.094974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:11.095825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 523/1062] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jul 21 17:22:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:11.096757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fa720-7f34-4313-be0c-76ba58eab771 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:11.161876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 540/1063] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:22:11.163396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba24475-2dcb-4f34-8390-b9c1b85aa043 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:11.195705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fa720-7f34-4313-be0c-76ba58eab771 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:11.211868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:11.230061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416d83-4993-4e65-bff9-14a223a50a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:11.230922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416d83-4993-4e65-bff9-14a223a50a0d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:11.231198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416d83-4993-4e65-bff9-14a223a50a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:11.231471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416d83-4993-4e65-bff9-14a223a50a0d None admin] PATH_INFO: `/v3/projects/bc7758b03780443bb9bc60fa2940f2b1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:11.232676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66416d83-4993-4e65-bff9-14a223a50a0d None admin] RBAC: Authorizing `identity:get_project(project_id=bc7758b03780443bb9bc60fa2940f2b1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:11.234115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66416d83-4993-4e65-bff9-14a223a50a0d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:11.236002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 524/1064] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:11 2022] GET /identity/v3/projects/bc7758b03780443bb9bc60fa2940f2b1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:11.236813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416d83-4993-4e65-bff9-14a223a50a0d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:11.256180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:11.269262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba24475-2dcb-4f34-8390-b9c1b85aa043 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:11.303783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a45dd2-354f-48df-ae60-898550071557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:11.304392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a45dd2-354f-48df-ae60-898550071557 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:11.304676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a45dd2-354f-48df-ae60-898550071557 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:11.304955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a45dd2-354f-48df-ae60-898550071557 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:11.305694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a45dd2-354f-48df-ae60-898550071557 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:11.305999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a45dd2-354f-48df-ae60-898550071557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:11.307379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a45dd2-354f-48df-ae60-898550071557 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:11.315824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 541/1065] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jul 21 17:22:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:11.316654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a45dd2-354f-48df-ae60-898550071557 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:11.344326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416d83-4993-4e65-bff9-14a223a50a0d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:11.376833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078ff89-3aca-4d4f-9ded-f0846fb45efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:11.377716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078ff89-3aca-4d4f-9ded-f0846fb45efd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:11.378067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078ff89-3aca-4d4f-9ded-f0846fb45efd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:11.378345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078ff89-3aca-4d4f-9ded-f0846fb45efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:11.389191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078ff89-3aca-4d4f-9ded-f0846fb45efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:11.390627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078ff89-3aca-4d4f-9ded-f0846fb45efd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-5078ff89-3aca-4d4f-9ded-f0846fb45efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:22:11.407733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:11.408686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:11.409807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:22:11.409807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:22:11.409807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:11.409807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:11.409807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 525/1066] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:11.410286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078ff89-3aca-4d4f-9ded-f0846fb45efd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:11.419785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a45dd2-354f-48df-ae60-898550071557 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:11.443939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:11.454038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938ba10-13c4-4fbd-b3ae-4ac815bfcd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:11.454686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938ba10-13c4-4fbd-b3ae-4ac815bfcd31 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:11.454963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938ba10-13c4-4fbd-b3ae-4ac815bfcd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:11.455235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938ba10-13c4-4fbd-b3ae-4ac815bfcd31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:11.456055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938ba10-13c4-4fbd-b3ae-4ac815bfcd31 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=bc7758b03780443bb9bc60fa2940f2b1, user.id=060fb4a9b1ac480fbc5a10270a2e49e6) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:11.456389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938ba10-13c4-4fbd-b3ae-4ac815bfcd31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:11.457801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938ba10-13c4-4fbd-b3ae-4ac815bfcd31 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:11.464050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 542/1067] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:22:11 2022] GET /identity/v3/role_assignments?user.id=060fb4a9b1ac480fbc5a10270a2e49e6&scope.project.id=bc7758b03780443bb9bc60fa2940f2b1&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:11.464690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938ba10-13c4-4fbd-b3ae-4ac815bfcd31 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:11.482027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:11.529771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078ff89-3aca-4d4f-9ded-f0846fb45efd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:11.572174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683a262-f764-403e-a1f3-12e2314755bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:11.573409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683a262-f764-403e-a1f3-12e2314755bc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:11.573524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683a262-f764-403e-a1f3-12e2314755bc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:11.573746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683a262-f764-403e-a1f3-12e2314755bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:11.586130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683a262-f764-403e-a1f3-12e2314755bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:11.587137 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938ba10-13c4-4fbd-b3ae-4ac815bfcd31 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:11.587602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683a262-f764-403e-a1f3-12e2314755bc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-9683a262-f764-403e-a1f3-12e2314755bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:22:11.606162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:11.607175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:11.608096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:22:11.608096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:22:11.608096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:11.608096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:11.608096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 526/1068] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:11.614799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683a262-f764-403e-a1f3-12e2314755bc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:11.631701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596208e-3d69-4008-a30f-d0d3bd90cf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:11.633154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596208e-3d69-4008-a30f-d0d3bd90cf98 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:11.633154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596208e-3d69-4008-a30f-d0d3bd90cf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:11.633437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596208e-3d69-4008-a30f-d0d3bd90cf98 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:11.634220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596208e-3d69-4008-a30f-d0d3bd90cf98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:11.634536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596208e-3d69-4008-a30f-d0d3bd90cf98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:11.635905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596208e-3d69-4008-a30f-d0d3bd90cf98 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:11.649003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 543/1069] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:11.717390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:11.732611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683a262-f764-403e-a1f3-12e2314755bc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:11.770494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb26825-57d4-4ff9-8855-5cc99adabe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:11.771176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb26825-57d4-4ff9-8855-5cc99adabe38 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:11.771463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb26825-57d4-4ff9-8855-5cc99adabe38 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:11.771740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb26825-57d4-4ff9-8855-5cc99adabe38 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:11.773655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb26825-57d4-4ff9-8855-5cc99adabe38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:11.773655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb26825-57d4-4ff9-8855-5cc99adabe38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:11.775051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb26825-57d4-4ff9-8855-5cc99adabe38 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:11.788651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 527/1070] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:11.794335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596208e-3d69-4008-a30f-d0d3bd90cf98 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:11.880933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:11.908742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596208e-3d69-4008-a30f-d0d3bd90cf98 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:11.947611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2cbc0-06a4-4353-9295-abdef4b5ec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:11.948680 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2cbc0-06a4-4353-9295-abdef4b5ec29 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:11.948958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2cbc0-06a4-4353-9295-abdef4b5ec29 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:11.949232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2cbc0-06a4-4353-9295-abdef4b5ec29 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:11.956470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2cbc0-06a4-4353-9295-abdef4b5ec29 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:11.957906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2cbc0-06a4-4353-9295-abdef4b5ec29 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f0a2cbc0-06a4-4353-9295-abdef4b5ec29 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:22:11.972098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:11.973285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:11.974587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 544/1071] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jul 21 17:22:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:11.979489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb26825-57d4-4ff9-8855-5cc99adabe38 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:12.080334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb26825-57d4-4ff9-8855-5cc99adabe38 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:12.113394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbc111-91dd-494a-821d-15af58dc8d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:12.114055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbc111-91dd-494a-821d-15af58dc8d44 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:12.114362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbc111-91dd-494a-821d-15af58dc8d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:12.114635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbc111-91dd-494a-821d-15af58dc8d44 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:12.115459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbc111-91dd-494a-821d-15af58dc8d44 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:12.115766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbc111-91dd-494a-821d-15af58dc8d44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:12.117872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbc111-91dd-494a-821d-15af58dc8d44 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:12.134564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 528/1072] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jul 21 17:22:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:12.140215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2cbc0-06a4-4353-9295-abdef4b5ec29 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:12.236560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2cbc0-06a4-4353-9295-abdef4b5ec29 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:12.269115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b688fd-847a-4b1f-a3dd-c0bdb1d67879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:12.269893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b688fd-847a-4b1f-a3dd-c0bdb1d67879 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:12.270170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b688fd-847a-4b1f-a3dd-c0bdb1d67879 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:12.270440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b688fd-847a-4b1f-a3dd-c0bdb1d67879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:12.272795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:12.281880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b688fd-847a-4b1f-a3dd-c0bdb1d67879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:12.283318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b688fd-847a-4b1f-a3dd-c0bdb1d67879 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-58b688fd-847a-4b1f-a3dd-c0bdb1d67879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:22:12.302555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:12.303627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:12.305508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:22:12.305508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:22:12.305508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:12.305508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:12.305508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 545/1073] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:12.309596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbc111-91dd-494a-821d-15af58dc8d44 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:12.318255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:12.412499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbc111-91dd-494a-821d-15af58dc8d44 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:12.445969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44074633-6d97-47b3-8a69-5aa77d844ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:12.446624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44074633-6d97-47b3-8a69-5aa77d844ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:12.446905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44074633-6d97-47b3-8a69-5aa77d844ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:12.447178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44074633-6d97-47b3-8a69-5aa77d844ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:12.448139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44074633-6d97-47b3-8a69-5aa77d844ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:12.448471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44074633-6d97-47b3-8a69-5aa77d844ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:12.449814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44074633-6d97-47b3-8a69-5aa77d844ac3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:12.461378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 529/1074] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:12.467085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b688fd-847a-4b1f-a3dd-c0bdb1d67879 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:12.564377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b688fd-847a-4b1f-a3dd-c0bdb1d67879 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:12.601318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd363b-1dc6-4c65-9f66-de32b22961c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:12.602121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd363b-1dc6-4c65-9f66-de32b22961c9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:12.602404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd363b-1dc6-4c65-9f66-de32b22961c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:12.602678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd363b-1dc6-4c65-9f66-de32b22961c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:12.609475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd363b-1dc6-4c65-9f66-de32b22961c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:12.610925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd363b-1dc6-4c65-9f66-de32b22961c9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-28cd363b-1dc6-4c65-9f66-de32b22961c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:22:12.624734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:12.625691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:12.626618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:22:12.626618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:22:12.626618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:22:12.626618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:22:12.626618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:22:12.626618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:22:12.626618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:22:12.626618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:12.626618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 546/1075] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:12.632431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44074633-6d97-47b3-8a69-5aa77d844ac3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:12.697343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:12.731778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44074633-6d97-47b3-8a69-5aa77d844ac3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:12.764404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d953c38-7eaa-465b-ba15-94aec9f680ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:12.765058 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d953c38-7eaa-465b-ba15-94aec9f680ea None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:12.765335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d953c38-7eaa-465b-ba15-94aec9f680ea None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:12.765607 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d953c38-7eaa-465b-ba15-94aec9f680ea None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:12.766384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d953c38-7eaa-465b-ba15-94aec9f680ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:12.766690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d953c38-7eaa-465b-ba15-94aec9f680ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:12.768127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d953c38-7eaa-465b-ba15-94aec9f680ea None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:12.780268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 530/1076] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:12.786097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd363b-1dc6-4c65-9f66-de32b22961c9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:12.831331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:12.883734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd363b-1dc6-4c65-9f66-de32b22961c9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:12.915829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28584b56-c3aa-4547-9d2f-19a41a1f3138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:12.916744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584b56-c3aa-4547-9d2f-19a41a1f3138 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:12.917024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584b56-c3aa-4547-9d2f-19a41a1f3138 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:12.917299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584b56-c3aa-4547-9d2f-19a41a1f3138 None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/380d6fe7057d44bfb20b0ae2bb1e664f/roles/085ea69bad9746939d7f6aad1904b3f3` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:12.919440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28584b56-c3aa-4547-9d2f-19a41a1f3138 None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=380d6fe7057d44bfb20b0ae2bb1e664f, role_id=085ea69bad9746939d7f6aad1904b3f3)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:12.920970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28584b56-c3aa-4547-9d2f-19a41a1f3138 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:12.934182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 547/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:22:12 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/380d6fe7057d44bfb20b0ae2bb1e664f/roles/085ea69bad9746939d7f6aad1904b3f3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:22:13.501177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75650401-b7c6-4aec-afc7-8b15047f0edc None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:13.501698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75650401-b7c6-4aec-afc7-8b15047f0edc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:13.501792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75650401-b7c6-4aec-afc7-8b15047f0edc None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:13.503104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 531/1078] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:13.508450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2ea3-3545-408e-9dfa-57e2db506aa6 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:13.508828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2ea3-3545-408e-9dfa-57e2db506aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:13.509122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2ea3-3545-408e-9dfa-57e2db506aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:13.525805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a60c2ea3-3545-408e-9dfa-57e2db506aa6 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:13.526989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a60c2ea3-3545-408e-9dfa-57e2db506aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:13.592383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 548/1079] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:13.617585 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75650401-b7c6-4aec-afc7-8b15047f0edc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:13.717450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75650401-b7c6-4aec-afc7-8b15047f0edc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:13.743171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:13.751933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a95838-d2e4-4d1d-9c1d-cbd6a7213b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:13.752695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95838-d2e4-4d1d-9c1d-cbd6a7213b59 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:13.752985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95838-d2e4-4d1d-9c1d-cbd6a7213b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:13.753265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95838-d2e4-4d1d-9c1d-cbd6a7213b59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:13.763388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a95838-d2e4-4d1d-9c1d-cbd6a7213b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-c2a95838-d2e4-4d1d-9c1d-cbd6a7213b59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:13.771086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:13.772737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:13.773934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 532/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:22:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:13.778900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2ea3-3545-408e-9dfa-57e2db506aa6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:13.813109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:13.880833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2ea3-3545-408e-9dfa-57e2db506aa6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:13.918715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627af41-1da6-4d8d-aef7-0861d717fbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:13.919390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627af41-1da6-4d8d-aef7-0861d717fbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:13.919673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627af41-1da6-4d8d-aef7-0861d717fbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:13.919950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627af41-1da6-4d8d-aef7-0861d717fbd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:13.920786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627af41-1da6-4d8d-aef7-0861d717fbd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:13.921094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627af41-1da6-4d8d-aef7-0861d717fbd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:13.922475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627af41-1da6-4d8d-aef7-0861d717fbd3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:13.935580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 549/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:22:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:13.936787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a95838-d2e4-4d1d-9c1d-cbd6a7213b59 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:14.043708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a95838-d2e4-4d1d-9c1d-cbd6a7213b59 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:14.076377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3427c-0079-4477-8994-9f425c8d297c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:14.077250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3427c-0079-4477-8994-9f425c8d297c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:14.077534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3427c-0079-4477-8994-9f425c8d297c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:14.077811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3427c-0079-4477-8994-9f425c8d297c None admin] PATH_INFO: `/v3/users/060fb4a9b1ac480fbc5a10270a2e49e6` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:14.078997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3427c-0079-4477-8994-9f425c8d297c None admin] RBAC: Authorizing `identity:get_user(user_id=060fb4a9b1ac480fbc5a10270a2e49e6)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:14.080470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3427c-0079-4477-8994-9f425c8d297c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:14.082119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 533/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:13 2022] GET /identity/v3/users/060fb4a9b1ac480fbc5a10270a2e49e6 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:14.088816 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627af41-1da6-4d8d-aef7-0861d717fbd3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:14.184090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:14.188317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627af41-1da6-4d8d-aef7-0861d717fbd3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:14.224067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2977b3-6450-4868-90db-f1b283b729ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:14.224948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2977b3-6450-4868-90db-f1b283b729ad None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:14.225564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2977b3-6450-4868-90db-f1b283b729ad None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:14.225564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2977b3-6450-4868-90db-f1b283b729ad None admin] PATH_INFO: `/v3/projects/bc7758b03780443bb9bc60fa2940f2b1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:14.226709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2977b3-6450-4868-90db-f1b283b729ad None admin] RBAC: Authorizing `identity:get_project(project_id=bc7758b03780443bb9bc60fa2940f2b1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:14.228144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2977b3-6450-4868-90db-f1b283b729ad None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:14.229892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 550/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:14 2022] GET /identity/v3/projects/bc7758b03780443bb9bc60fa2940f2b1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:14.235971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3427c-0079-4477-8994-9f425c8d297c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:14.290611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:14.335899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3427c-0079-4477-8994-9f425c8d297c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:14.368438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c3e0c-2749-4d74-9d05-b3fc6f2c54f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:14.369323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c3e0c-2749-4d74-9d05-b3fc6f2c54f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:14.369611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c3e0c-2749-4d74-9d05-b3fc6f2c54f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:14.369878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c3e0c-2749-4d74-9d05-b3fc6f2c54f3 None admin] PATH_INFO: `/v3/projects/bc7758b03780443bb9bc60fa2940f2b1/users/060fb4a9b1ac480fbc5a10270a2e49e6/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:14.371846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c3e0c-2749-4d74-9d05-b3fc6f2c54f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7758b03780443bb9bc60fa2940f2b1, user_id=060fb4a9b1ac480fbc5a10270a2e49e6, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:14.373299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c3e0c-2749-4d74-9d05-b3fc6f2c54f3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:14.388149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 534/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:22:14 2022] PUT /identity/v3/projects/bc7758b03780443bb9bc60fa2940f2b1/users/060fb4a9b1ac480fbc5a10270a2e49e6/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:22:14.939018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf859d-1407-4252-a0b5-3ce9b06ac8d4 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:14.939457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf859d-1407-4252-a0b5-3ce9b06ac8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:14.939622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf859d-1407-4252-a0b5-3ce9b06ac8d4 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:14.941024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 551/1085] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:14.946894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0290b-5dfc-472c-98b3-f50bdb4f79b6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:14.947318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0290b-5dfc-472c-98b3-f50bdb4f79b6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:14.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0290b-5dfc-472c-98b3-f50bdb4f79b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:14.964822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-dda0290b-5dfc-472c-98b3-f50bdb4f79b6 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:14.966008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-dda0290b-5dfc-472c-98b3-f50bdb4f79b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:15.033895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 535/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:15.057625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf859d-1407-4252-a0b5-3ce9b06ac8d4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:15.158371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf859d-1407-4252-a0b5-3ce9b06ac8d4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:15.190386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e99aac-d517-467e-b256-e2eaa24728f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:15.191119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e99aac-d517-467e-b256-e2eaa24728f2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:15.191400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e99aac-d517-467e-b256-e2eaa24728f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:15.191669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e99aac-d517-467e-b256-e2eaa24728f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:15.202793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e99aac-d517-467e-b256-e2eaa24728f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-31e99aac-d517-467e-b256-e2eaa24728f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:15.211432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:15.212482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:15.213317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 552/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Jul 21 17:22:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:15.218688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0290b-5dfc-472c-98b3-f50bdb4f79b6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:15.262836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:15.318451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0290b-5dfc-472c-98b3-f50bdb4f79b6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:15.352199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad2694-e5a5-484a-b7cf-d324cd8bd3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:15.352933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad2694-e5a5-484a-b7cf-d324cd8bd3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:15.353218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad2694-e5a5-484a-b7cf-d324cd8bd3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:15.353495 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad2694-e5a5-484a-b7cf-d324cd8bd3a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:15.354248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad2694-e5a5-484a-b7cf-d324cd8bd3a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:15.354555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad2694-e5a5-484a-b7cf-d324cd8bd3a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:15.355912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad2694-e5a5-484a-b7cf-d324cd8bd3a8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:15.363725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 536/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jul 21 17:22:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:15.370068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e99aac-d517-467e-b256-e2eaa24728f2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:15.397088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:15.469048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e99aac-d517-467e-b256-e2eaa24728f2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:15.503065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b82d88-1e1f-4812-b11c-d5f63725d497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:15.503866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b82d88-1e1f-4812-b11c-d5f63725d497 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:15.504145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b82d88-1e1f-4812-b11c-d5f63725d497 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:15.504451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b82d88-1e1f-4812-b11c-d5f63725d497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:15.515894 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b82d88-1e1f-4812-b11c-d5f63725d497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:15.517378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b82d88-1e1f-4812-b11c-d5f63725d497 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-37b82d88-1e1f-4812-b11c-d5f63725d497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:22:15.534795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:15.535855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:15.536958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:22:15.536958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:22:15.536958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:15.536958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:15.536958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 553/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:15.541982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad2694-e5a5-484a-b7cf-d324cd8bd3a8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:15.643264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad2694-e5a5-484a-b7cf-d324cd8bd3a8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:15.650270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:15.677500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d551d9-a175-4942-80f3-b4d0b6aecd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:15.678221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d551d9-a175-4942-80f3-b4d0b6aecd7a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:15.679080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d551d9-a175-4942-80f3-b4d0b6aecd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:15.679453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d551d9-a175-4942-80f3-b4d0b6aecd7a None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:15.680180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d551d9-a175-4942-80f3-b4d0b6aecd7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:15.680527 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d551d9-a175-4942-80f3-b4d0b6aecd7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:15.681887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d551d9-a175-4942-80f3-b4d0b6aecd7a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:15.693553 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 537/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:15.699482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b82d88-1e1f-4812-b11c-d5f63725d497 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:15.775444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:15.797080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b82d88-1e1f-4812-b11c-d5f63725d497 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:15.830165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10139954-ffbd-43d0-98bd-7c8b69d768d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:15.830944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10139954-ffbd-43d0-98bd-7c8b69d768d4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:15.831226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10139954-ffbd-43d0-98bd-7c8b69d768d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:15.831501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10139954-ffbd-43d0-98bd-7c8b69d768d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:15.838849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10139954-ffbd-43d0-98bd-7c8b69d768d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:15.840318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10139954-ffbd-43d0-98bd-7c8b69d768d4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-10139954-ffbd-43d0-98bd-7c8b69d768d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:22:15.854562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:15.855503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:15.856503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 554/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Jul 21 17:22:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:22:15.861450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d551d9-a175-4942-80f3-b4d0b6aecd7a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:15.962404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d551d9-a175-4942-80f3-b4d0b6aecd7a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:15.996086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcd0fe-d37c-4d63-9f02-bfcd18d618ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:15.996805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcd0fe-d37c-4d63-9f02-bfcd18d618ad None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:15.997086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcd0fe-d37c-4d63-9f02-bfcd18d618ad None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:15.997357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcd0fe-d37c-4d63-9f02-bfcd18d618ad None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:15.998181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcd0fe-d37c-4d63-9f02-bfcd18d618ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:15.998489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcd0fe-d37c-4d63-9f02-bfcd18d618ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:15.999897 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcd0fe-d37c-4d63-9f02-bfcd18d618ad None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:16.016634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 538/1092] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Jul 21 17:22:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:16.022966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10139954-ffbd-43d0-98bd-7c8b69d768d4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:16.120804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10139954-ffbd-43d0-98bd-7c8b69d768d4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:16.153664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fd02c-7afe-4163-9194-d4e29affe774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:16.154315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fd02c-7afe-4163-9194-d4e29affe774 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:16.154608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fd02c-7afe-4163-9194-d4e29affe774 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:16.154887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fd02c-7afe-4163-9194-d4e29affe774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:16.164148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:16.223174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fd02c-7afe-4163-9194-d4e29affe774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:16.224685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fd02c-7afe-4163-9194-d4e29affe774 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:16.326309 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 555/1093] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 17:22:16.332027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcd0fe-d37c-4d63-9f02-bfcd18d618ad None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:16.357487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:16.435466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcd0fe-d37c-4d63-9f02-bfcd18d618ad None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:16.463306 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce59a76-a90b-43d5-9c78-b30259b85f8a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:16.463599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce59a76-a90b-43d5-9c78-b30259b85f8a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:16.463887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce59a76-a90b-43d5-9c78-b30259b85f8a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:16.465293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 556/1094] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:22:16.469194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24947da0-de82-44e5-89b1-6aaaedfe13bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:16.469990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24947da0-de82-44e5-89b1-6aaaedfe13bb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:16.470107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee6448-6539-450f-8ef6-342b0f91347c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:16.470274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24947da0-de82-44e5-89b1-6aaaedfe13bb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:16.470368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee6448-6539-450f-8ef6-342b0f91347c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:16.470543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24947da0-de82-44e5-89b1-6aaaedfe13bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:16.470636 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee6448-6539-450f-8ef6-342b0f91347c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:16.481472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24947da0-de82-44e5-89b1-6aaaedfe13bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:16.482928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24947da0-de82-44e5-89b1-6aaaedfe13bb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:16.487569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f3ee6448-6539-450f-8ef6-342b0f91347c None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:16.488785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f3ee6448-6539-450f-8ef6-342b0f91347c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-24947da0-de82-44e5-89b1-6aaaedfe13bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:22:16.500984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:16.502065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:16.503062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:22:16.503062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:22:16.503062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:22:16.503062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:16.503062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 539/1095] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:22:16.508061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24947da0-de82-44e5-89b1-6aaaedfe13bb None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:16.519430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:16.557997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 557/1096] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:22:16.581225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee6448-6539-450f-8ef6-342b0f91347c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:16.595512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:16.621696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24947da0-de82-44e5-89b1-6aaaedfe13bb None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:16.654596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a2209-a302-44a3-9def-5651d863d72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:16.655252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a2209-a302-44a3-9def-5651d863d72a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:16.655527 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a2209-a302-44a3-9def-5651d863d72a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:16.655795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a2209-a302-44a3-9def-5651d863d72a None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:16.656536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a2209-a302-44a3-9def-5651d863d72a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:16.656851 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a2209-a302-44a3-9def-5651d863d72a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:16.658191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a2209-a302-44a3-9def-5651d863d72a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:16.669486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 540/1097] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:16.675387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a2209-a302-44a3-9def-5651d863d72a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:16.680666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee6448-6539-450f-8ef6-342b0f91347c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:16.713843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bd0e1-5a9e-474e-95d9-c4a24957e958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:16.714561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bd0e1-5a9e-474e-95d9-c4a24957e958 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:16.714845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bd0e1-5a9e-474e-95d9-c4a24957e958 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:16.715119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bd0e1-5a9e-474e-95d9-c4a24957e958 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:16.725759 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bd0e1-5a9e-474e-95d9-c4a24957e958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ae2bd0e1-5a9e-474e-95d9-c4a24957e958 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:16.733393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:16.734471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:16.735307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 558/1098] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:22:16.740459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bd0e1-5a9e-474e-95d9-c4a24957e958 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:16.768050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:16.774582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a2209-a302-44a3-9def-5651d863d72a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:16.808100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c15f64-96aa-4a65-bac5-a1e300b5c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:16.808924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c15f64-96aa-4a65-bac5-a1e300b5c53e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:16.809208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c15f64-96aa-4a65-bac5-a1e300b5c53e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:16.809483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c15f64-96aa-4a65-bac5-a1e300b5c53e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:16.812445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:16.817332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c15f64-96aa-4a65-bac5-a1e300b5c53e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:16.818993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c15f64-96aa-4a65-bac5-a1e300b5c53e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-d3c15f64-96aa-4a65-bac5-a1e300b5c53e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:22:16.833978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:16.835508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:16.836432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:22:16.836432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:22:16.836432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:22:16.836432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:22:16.836432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:22:16.836432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:22:16.836432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:22:16.836432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:16.836432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 541/1099] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:22:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:22:16.841028 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bd0e1-5a9e-474e-95d9-c4a24957e958 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:16.841490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c15f64-96aa-4a65-bac5-a1e300b5c53e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:16.873993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20493178-356e-4428-b27a-074d63139c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:16.874660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20493178-356e-4428-b27a-074d63139c38 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:16.874941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20493178-356e-4428-b27a-074d63139c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:16.875218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20493178-356e-4428-b27a-074d63139c38 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:16.875966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20493178-356e-4428-b27a-074d63139c38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:16.876274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20493178-356e-4428-b27a-074d63139c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:16.877667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20493178-356e-4428-b27a-074d63139c38 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:16.885793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 559/1100] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:22:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:16.891374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20493178-356e-4428-b27a-074d63139c38 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:16.943432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c15f64-96aa-4a65-bac5-a1e300b5c53e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:16.978332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd59113-40e2-4c3d-a797-bad726321684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:16.978951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd59113-40e2-4c3d-a797-bad726321684 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:16.979177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd59113-40e2-4c3d-a797-bad726321684 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:16.979450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd59113-40e2-4c3d-a797-bad726321684 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:16.980234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd59113-40e2-4c3d-a797-bad726321684 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:16.980569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd59113-40e2-4c3d-a797-bad726321684 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:16.981984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd59113-40e2-4c3d-a797-bad726321684 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:16.985350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:16.994204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 542/1101] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:22:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:16.996398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20493178-356e-4428-b27a-074d63139c38 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:17.000311 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd59113-40e2-4c3d-a797-bad726321684 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:17.029821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219df551-c488-4219-a413-c805f39f24eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:17.030690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219df551-c488-4219-a413-c805f39f24eb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:17.030976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219df551-c488-4219-a413-c805f39f24eb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:17.031252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219df551-c488-4219-a413-c805f39f24eb None admin] PATH_INFO: `/v3/users/060fb4a9b1ac480fbc5a10270a2e49e6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:17.032475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219df551-c488-4219-a413-c805f39f24eb None admin] RBAC: Authorizing `identity:get_user(user_id=060fb4a9b1ac480fbc5a10270a2e49e6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:17.033905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219df551-c488-4219-a413-c805f39f24eb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:17.040145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 560/1102] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:16 2022] GET /identity/v3/users/060fb4a9b1ac480fbc5a10270a2e49e6 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:17.041168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219df551-c488-4219-a413-c805f39f24eb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:17.056327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:17.099587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd59113-40e2-4c3d-a797-bad726321684 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:17.133555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f463428-433b-40e2-a563-8108acc6aabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:17.134213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f463428-433b-40e2-a563-8108acc6aabf None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:17.134496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f463428-433b-40e2-a563-8108acc6aabf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:17.134803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f463428-433b-40e2-a563-8108acc6aabf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:17.135633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f463428-433b-40e2-a563-8108acc6aabf None admin] RBAC: Adding query filter params (role.id=085ea69bad9746939d7f6aad1904b3f3, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=380d6fe7057d44bfb20b0ae2bb1e664f) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:17.135949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f463428-433b-40e2-a563-8108acc6aabf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:17.137396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f463428-433b-40e2-a563-8108acc6aabf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:17.139619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219df551-c488-4219-a413-c805f39f24eb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:17.143402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 543/1103] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:22:16 2022] GET /identity/v3/role_assignments?user.id=380d6fe7057d44bfb20b0ae2bb1e664f&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=085ea69bad9746939d7f6aad1904b3f3 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:17.176769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aeedd-95b0-4fd6-a477-18ae0ea21b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:17.177442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aeedd-95b0-4fd6-a477-18ae0ea21b47 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:17.177726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aeedd-95b0-4fd6-a477-18ae0ea21b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:17.178003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aeedd-95b0-4fd6-a477-18ae0ea21b47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:17.210717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:17.261069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aeedd-95b0-4fd6-a477-18ae0ea21b47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:17.262447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aeedd-95b0-4fd6-a477-18ae0ea21b47 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:17.365289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 561/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:22:17.371256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f463428-433b-40e2-a563-8108acc6aabf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:17.473465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f463428-433b-40e2-a563-8108acc6aabf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:17.493728 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:17.508205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc03ddf-ea7d-4549-98d0-26817d5c2fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:17.509094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc03ddf-ea7d-4549-98d0-26817d5c2fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:17.509371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc03ddf-ea7d-4549-98d0-26817d5c2fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:17.509641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc03ddf-ea7d-4549-98d0-26817d5c2fb2 None admin] PATH_INFO: `/v3/projects/bc7758b03780443bb9bc60fa2940f2b1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:17.510806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc03ddf-ea7d-4549-98d0-26817d5c2fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7758b03780443bb9bc60fa2940f2b1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:17.512379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc03ddf-ea7d-4549-98d0-26817d5c2fb2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:17.513956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 544/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:17 2022] GET /identity/v3/projects/bc7758b03780443bb9bc60fa2940f2b1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:17.520344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aeedd-95b0-4fd6-a477-18ae0ea21b47 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:17.581544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:17.626239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aeedd-95b0-4fd6-a477-18ae0ea21b47 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:17.661419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ae024-4ea4-43cb-bd0f-e4118a98acd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:17.662097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ae024-4ea4-43cb-bd0f-e4118a98acd7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:17.662404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ae024-4ea4-43cb-bd0f-e4118a98acd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:17.662710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ae024-4ea4-43cb-bd0f-e4118a98acd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:17.663669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ae024-4ea4-43cb-bd0f-e4118a98acd7 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=bc7758b03780443bb9bc60fa2940f2b1, user.id=060fb4a9b1ac480fbc5a10270a2e49e6) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:17.663979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ae024-4ea4-43cb-bd0f-e4118a98acd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:17.665797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ae024-4ea4-43cb-bd0f-e4118a98acd7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:17.671697 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 562/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:22:17 2022] GET /identity/v3/role_assignments?user.id=060fb4a9b1ac480fbc5a10270a2e49e6&scope.project.id=bc7758b03780443bb9bc60fa2940f2b1&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:19.706125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12fb27-70ec-4b1d-8949-c71e185dcde7 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:19.706645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12fb27-70ec-4b1d-8949-c71e185dcde7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:19.706760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12fb27-70ec-4b1d-8949-c71e185dcde7 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:19.708096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 545/1107] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:19.713505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9d974-9cf1-4a65-8f97-862917b0c9eb None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:19.713895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9d974-9cf1-4a65-8f97-862917b0c9eb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:19.714063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9d974-9cf1-4a65-8f97-862917b0c9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:19.730829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-19e9d974-9cf1-4a65-8f97-862917b0c9eb None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:19.732001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-19e9d974-9cf1-4a65-8f97-862917b0c9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:19.796991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 563/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:19.820923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12fb27-70ec-4b1d-8949-c71e185dcde7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:19.919953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12fb27-70ec-4b1d-8949-c71e185dcde7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:19.952316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9e872-99eb-49db-ad04-4824911b1959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:19.953099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9e872-99eb-49db-ad04-4824911b1959 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:19.953387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9e872-99eb-49db-ad04-4824911b1959 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:19.953659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9e872-99eb-49db-ad04-4824911b1959 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:19.954814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9e872-99eb-49db-ad04-4824911b1959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:19.956285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9e872-99eb-49db-ad04-4824911b1959 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:19.957867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 546/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:19.964170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9d974-9cf1-4a65-8f97-862917b0c9eb None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:20.015439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:20.061740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9d974-9cf1-4a65-8f97-862917b0c9eb None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:20.094621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0335d-4705-432a-9fa5-30b762ab335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:20.095282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0335d-4705-432a-9fa5-30b762ab335d None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:20.095567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0335d-4705-432a-9fa5-30b762ab335d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:20.095844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0335d-4705-432a-9fa5-30b762ab335d None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:20.096572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0335d-4705-432a-9fa5-30b762ab335d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:20.098001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0335d-4705-432a-9fa5-30b762ab335d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:20.117655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 564/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:22:19 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:22:22.145525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e73147-7f03-4898-a3d1-4b40bba61266 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:22.146087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e73147-7f03-4898-a3d1-4b40bba61266 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:22.146087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e73147-7f03-4898-a3d1-4b40bba61266 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:22.147971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 547/1111] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:22.153029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c40f8-5514-42d3-8d7a-580191849f92 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:22.153430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c40f8-5514-42d3-8d7a-580191849f92 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:22.153580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c40f8-5514-42d3-8d7a-580191849f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:22.170357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-097c40f8-5514-42d3-8d7a-580191849f92 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:22.171524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-097c40f8-5514-42d3-8d7a-580191849f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:22.235934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 565/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:22:22.258886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e73147-7f03-4898-a3d1-4b40bba61266 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:22.315532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:22.359643 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e73147-7f03-4898-a3d1-4b40bba61266 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:22.391186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e30d2-4c08-4b17-bacf-3aa22a97a015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:22.392027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e30d2-4c08-4b17-bacf-3aa22a97a015 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:22.392323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e30d2-4c08-4b17-bacf-3aa22a97a015 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:22.392597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e30d2-4c08-4b17-bacf-3aa22a97a015 None admin] PATH_INFO: `/v3/roles/110c20a9ff1e455d8c43ffdfbdd1447b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:22.394056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e30d2-4c08-4b17-bacf-3aa22a97a015 None admin] RBAC: Authorizing `identity:get_role(role_id=110c20a9ff1e455d8c43ffdfbdd1447b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:22.395392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e30d2-4c08-4b17-bacf-3aa22a97a015 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:22.397354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 548/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:22 2022] GET /identity/v3/roles/110c20a9ff1e455d8c43ffdfbdd1447b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:22.403138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c40f8-5514-42d3-8d7a-580191849f92 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:22.442809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:22.501631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c40f8-5514-42d3-8d7a-580191849f92 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:22.534188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f4c9b-8945-4311-85f1-44c7c1567e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:22.535046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f4c9b-8945-4311-85f1-44c7c1567e61 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:22.535324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f4c9b-8945-4311-85f1-44c7c1567e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:22.535594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f4c9b-8945-4311-85f1-44c7c1567e61 None admin] PATH_INFO: `/v3/users/e6d33353a3064b6eaf1e2c4e2d9547ce` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:22.548777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f4c9b-8945-4311-85f1-44c7c1567e61 None admin] RBAC: Authorizing `identity:get_user(user_id=e6d33353a3064b6eaf1e2c4e2d9547ce)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:22.550207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f4c9b-8945-4311-85f1-44c7c1567e61 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:22.551812 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 566/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:22 2022] GET /identity/v3/users/e6d33353a3064b6eaf1e2c4e2d9547ce => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:22.557474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e30d2-4c08-4b17-bacf-3aa22a97a015 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:22.653399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e30d2-4c08-4b17-bacf-3aa22a97a015 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:22.674914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:22.685654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73621a8a-0c10-41c5-a3d1-000824a1e30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:22.686318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73621a8a-0c10-41c5-a3d1-000824a1e30c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:22.686595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73621a8a-0c10-41c5-a3d1-000824a1e30c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:22.686865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73621a8a-0c10-41c5-a3d1-000824a1e30c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:22.751268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73621a8a-0c10-41c5-a3d1-000824a1e30c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:22.752760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73621a8a-0c10-41c5-a3d1-000824a1e30c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:22.845810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 549/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:22:22.851886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f4c9b-8945-4311-85f1-44c7c1567e61 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:22.944074 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:22.950646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f4c9b-8945-4311-85f1-44c7c1567e61 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:22.983038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca0ccc-9efc-4ec7-99af-a8a96905ce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:22.983882 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca0ccc-9efc-4ec7-99af-a8a96905ce91 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:22.984167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca0ccc-9efc-4ec7-99af-a8a96905ce91 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:22.984475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca0ccc-9efc-4ec7-99af-a8a96905ce91 None admin] PATH_INFO: `/v3/projects/bc7758b03780443bb9bc60fa2940f2b1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:22.985632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca0ccc-9efc-4ec7-99af-a8a96905ce91 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7758b03780443bb9bc60fa2940f2b1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:22.987093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca0ccc-9efc-4ec7-99af-a8a96905ce91 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:22.988795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 567/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:22 2022] GET /identity/v3/projects/bc7758b03780443bb9bc60fa2940f2b1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:22.995353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73621a8a-0c10-41c5-a3d1-000824a1e30c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:23.019544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:23.092229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73621a8a-0c10-41c5-a3d1-000824a1e30c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:23.125208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0055f-3867-4e59-b190-0deee65bfa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:23.125847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0055f-3867-4e59-b190-0deee65bfa1f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:23.126155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0055f-3867-4e59-b190-0deee65bfa1f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:23.126432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0055f-3867-4e59-b190-0deee65bfa1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:23.127254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0055f-3867-4e59-b190-0deee65bfa1f None admin] RBAC: Adding query filter params (role.id=110c20a9ff1e455d8c43ffdfbdd1447b, scope.project.id=bc7758b03780443bb9bc60fa2940f2b1, user.id=e6d33353a3064b6eaf1e2c4e2d9547ce) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:23.127563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0055f-3867-4e59-b190-0deee65bfa1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:23.129000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0055f-3867-4e59-b190-0deee65bfa1f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:23.134495 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 550/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:22:22 2022] GET /identity/v3/role_assignments?user.id=e6d33353a3064b6eaf1e2c4e2d9547ce&scope.project.id=bc7758b03780443bb9bc60fa2940f2b1&role.id=110c20a9ff1e455d8c43ffdfbdd1447b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:25.154405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d9d29-3edb-4d4a-9828-5015eb8c5390 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:25.154980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d9d29-3edb-4d4a-9828-5015eb8c5390 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:25.154980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d9d29-3edb-4d4a-9828-5015eb8c5390 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:25.156347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 568/1118] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:25.161471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d018c-c647-4cca-a1d6-9c36106a65a0 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:25.161900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d018c-c647-4cca-a1d6-9c36106a65a0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:25.162065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d018c-c647-4cca-a1d6-9c36106a65a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:25.179014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5f1d018c-c647-4cca-a1d6-9c36106a65a0 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:25.180200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5f1d018c-c647-4cca-a1d6-9c36106a65a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:25.244517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 551/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:25.266871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d9d29-3edb-4d4a-9828-5015eb8c5390 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:25.367793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d9d29-3edb-4d4a-9828-5015eb8c5390 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:25.400670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c7657-ba78-401f-a25e-05fae23524f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:25.401532 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c7657-ba78-401f-a25e-05fae23524f7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:25.401814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c7657-ba78-401f-a25e-05fae23524f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:25.402088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c7657-ba78-401f-a25e-05fae23524f7 None admin] PATH_INFO: `/v3/roles/110c20a9ff1e455d8c43ffdfbdd1447b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:25.403541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c7657-ba78-401f-a25e-05fae23524f7 None admin] RBAC: Authorizing `identity:get_role(role_id=110c20a9ff1e455d8c43ffdfbdd1447b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:25.404934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c7657-ba78-401f-a25e-05fae23524f7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:25.406288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 569/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:25 2022] GET /identity/v3/roles/110c20a9ff1e455d8c43ffdfbdd1447b => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:25.412071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d018c-c647-4cca-a1d6-9c36106a65a0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:25.429933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:25.510654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d018c-c647-4cca-a1d6-9c36106a65a0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:25.543615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb2eb1-194a-49a3-ad73-3c0ec24774fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:25.544502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb2eb1-194a-49a3-ad73-3c0ec24774fb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:25.544787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb2eb1-194a-49a3-ad73-3c0ec24774fb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:25.545067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb2eb1-194a-49a3-ad73-3c0ec24774fb None admin] PATH_INFO: `/v3/users/e6d33353a3064b6eaf1e2c4e2d9547ce` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:25.546239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb2eb1-194a-49a3-ad73-3c0ec24774fb None admin] RBAC: Authorizing `identity:get_user(user_id=e6d33353a3064b6eaf1e2c4e2d9547ce)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:25.547666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb2eb1-194a-49a3-ad73-3c0ec24774fb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:25.549322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 552/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:25 2022] GET /identity/v3/users/e6d33353a3064b6eaf1e2c4e2d9547ce => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:25.554971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c7657-ba78-401f-a25e-05fae23524f7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:25.586148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:25.653773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c7657-ba78-401f-a25e-05fae23524f7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:25.685964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537a4b7-59b3-43cf-bacb-56811d53ce06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:25.686803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537a4b7-59b3-43cf-bacb-56811d53ce06 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:25.687081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537a4b7-59b3-43cf-bacb-56811d53ce06 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:25.687351 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537a4b7-59b3-43cf-bacb-56811d53ce06 None admin] PATH_INFO: `/v3/projects/bc7758b03780443bb9bc60fa2940f2b1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:25.688525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537a4b7-59b3-43cf-bacb-56811d53ce06 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7758b03780443bb9bc60fa2940f2b1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:25.689942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537a4b7-59b3-43cf-bacb-56811d53ce06 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:25.691599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 570/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:25 2022] GET /identity/v3/projects/bc7758b03780443bb9bc60fa2940f2b1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:25.698984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb2eb1-194a-49a3-ad73-3c0ec24774fb None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:25.780805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:25.795523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb2eb1-194a-49a3-ad73-3c0ec24774fb None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:25.828575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcf988-efc4-4497-b3d1-a71213130471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:25.829443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcf988-efc4-4497-b3d1-a71213130471 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:25.829721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcf988-efc4-4497-b3d1-a71213130471 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:25.829992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcf988-efc4-4497-b3d1-a71213130471 None admin] PATH_INFO: `/v3/projects/bc7758b03780443bb9bc60fa2940f2b1/users/e6d33353a3064b6eaf1e2c4e2d9547ce/roles/110c20a9ff1e455d8c43ffdfbdd1447b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:25.832124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddcf988-efc4-4497-b3d1-a71213130471 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7758b03780443bb9bc60fa2940f2b1, user_id=e6d33353a3064b6eaf1e2c4e2d9547ce, role_id=110c20a9ff1e455d8c43ffdfbdd1447b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:25.833572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddcf988-efc4-4497-b3d1-a71213130471 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:25.845640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 553/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:22:25 2022] PUT /identity/v3/projects/bc7758b03780443bb9bc60fa2940f2b1/users/e6d33353a3064b6eaf1e2c4e2d9547ce/roles/110c20a9ff1e455d8c43ffdfbdd1447b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:22:27.826310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6509d-fac4-4db3-8c4e-4ea5bfe72db1 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:27.826863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6509d-fac4-4db3-8c4e-4ea5bfe72db1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:27.826863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6509d-fac4-4db3-8c4e-4ea5bfe72db1 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:27.828203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 571/1124] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:27.833416 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a84562-41dd-42d7-aedf-7d8d04bde813 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:27.833862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a84562-41dd-42d7-aedf-7d8d04bde813 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:27.834036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a84562-41dd-42d7-aedf-7d8d04bde813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:27.850720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-63a84562-41dd-42d7-aedf-7d8d04bde813 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:27.851879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-63a84562-41dd-42d7-aedf-7d8d04bde813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:27.915735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 554/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:27.939414 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6509d-fac4-4db3-8c4e-4ea5bfe72db1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:28.036488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:28.041511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6509d-fac4-4db3-8c4e-4ea5bfe72db1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:28.074004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0c46e-fa35-43b7-aed6-9161654752ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:28.074842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0c46e-fa35-43b7-aed6-9161654752ce None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:28.075121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0c46e-fa35-43b7-aed6-9161654752ce None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:28.075393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0c46e-fa35-43b7-aed6-9161654752ce None admin] PATH_INFO: `/v3/roles/110c20a9ff1e455d8c43ffdfbdd1447b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:28.076917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0c46e-fa35-43b7-aed6-9161654752ce None admin] RBAC: Authorizing `identity:get_role(role_id=110c20a9ff1e455d8c43ffdfbdd1447b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:28.078230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0c46e-fa35-43b7-aed6-9161654752ce None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:28.079589 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 572/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:27 2022] GET /identity/v3/roles/110c20a9ff1e455d8c43ffdfbdd1447b => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:28.086392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a84562-41dd-42d7-aedf-7d8d04bde813 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:28.160629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:28.184189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a84562-41dd-42d7-aedf-7d8d04bde813 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:28.217475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f04f44-4f06-4ddd-bf08-d1e25ccaec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:28.218387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f04f44-4f06-4ddd-bf08-d1e25ccaec4d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:28.218674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f04f44-4f06-4ddd-bf08-d1e25ccaec4d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:28.218953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f04f44-4f06-4ddd-bf08-d1e25ccaec4d None admin] PATH_INFO: `/v3/users/e6d33353a3064b6eaf1e2c4e2d9547ce` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:28.220125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f04f44-4f06-4ddd-bf08-d1e25ccaec4d None admin] RBAC: Authorizing `identity:get_user(user_id=e6d33353a3064b6eaf1e2c4e2d9547ce)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:28.221579 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f04f44-4f06-4ddd-bf08-d1e25ccaec4d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:28.223196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 555/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:28 2022] GET /identity/v3/users/e6d33353a3064b6eaf1e2c4e2d9547ce => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:28.229397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0c46e-fa35-43b7-aed6-9161654752ce None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:28.324488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0c46e-fa35-43b7-aed6-9161654752ce None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:28.357654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079857dd-cd02-44f6-a2d8-d2375dd844f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:28.358412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079857dd-cd02-44f6-a2d8-d2375dd844f7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:28.358693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079857dd-cd02-44f6-a2d8-d2375dd844f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:28.358968 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079857dd-cd02-44f6-a2d8-d2375dd844f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:28.393235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:28.425416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079857dd-cd02-44f6-a2d8-d2375dd844f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:28.426853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079857dd-cd02-44f6-a2d8-d2375dd844f7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:28.534147 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 573/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:22:28.540045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f04f44-4f06-4ddd-bf08-d1e25ccaec4d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:28.634713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f04f44-4f06-4ddd-bf08-d1e25ccaec4d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:28.667348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a622fa-ac07-4abe-bfcb-b9d596315779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:28.668185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a622fa-ac07-4abe-bfcb-b9d596315779 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:28.668491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a622fa-ac07-4abe-bfcb-b9d596315779 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:28.668767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a622fa-ac07-4abe-bfcb-b9d596315779 None admin] PATH_INFO: `/v3/projects/bc7758b03780443bb9bc60fa2940f2b1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:28.669926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a622fa-ac07-4abe-bfcb-b9d596315779 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7758b03780443bb9bc60fa2940f2b1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:28.671326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a622fa-ac07-4abe-bfcb-b9d596315779 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:28.673009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 556/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:28 2022] GET /identity/v3/projects/bc7758b03780443bb9bc60fa2940f2b1 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:28.679121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079857dd-cd02-44f6-a2d8-d2375dd844f7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:28.743576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:28.775704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079857dd-cd02-44f6-a2d8-d2375dd844f7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:28.808220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75780fc8-45aa-457c-bbb5-736d0126a22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:28.808884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75780fc8-45aa-457c-bbb5-736d0126a22f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:28.809167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75780fc8-45aa-457c-bbb5-736d0126a22f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:28.809441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75780fc8-45aa-457c-bbb5-736d0126a22f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:28.810259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75780fc8-45aa-457c-bbb5-736d0126a22f None admin] RBAC: Adding query filter params (role.id=110c20a9ff1e455d8c43ffdfbdd1447b, scope.project.id=bc7758b03780443bb9bc60fa2940f2b1, user.id=e6d33353a3064b6eaf1e2c4e2d9547ce) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:28.810571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75780fc8-45aa-457c-bbb5-736d0126a22f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:28.811976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75780fc8-45aa-457c-bbb5-736d0126a22f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:28.817616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 574/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:22:28 2022] GET /identity/v3/role_assignments?user.id=e6d33353a3064b6eaf1e2c4e2d9547ce&scope.project.id=bc7758b03780443bb9bc60fa2940f2b1&role.id=110c20a9ff1e455d8c43ffdfbdd1447b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:30.844789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b64ea3-c5cb-4a14-a8e0-53b9ff3aad59 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:30.845319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b64ea3-c5cb-4a14-a8e0-53b9ff3aad59 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:30.845319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b64ea3-c5cb-4a14-a8e0-53b9ff3aad59 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:30.846694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 557/1131] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:30.853933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda341-4ab0-48af-a15b-15bd59dc029d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:30.854354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda341-4ab0-48af-a15b-15bd59dc029d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:30.854482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda341-4ab0-48af-a15b-15bd59dc029d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:30.871568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-21dda341-4ab0-48af-a15b-15bd59dc029d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:30.872773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-21dda341-4ab0-48af-a15b-15bd59dc029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:30.939377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 575/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:30.963399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b64ea3-c5cb-4a14-a8e0-53b9ff3aad59 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:30.978467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:31.070714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b64ea3-c5cb-4a14-a8e0-53b9ff3aad59 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:31.104038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521c24b-3aa9-4364-8f8c-a06d3d8b9bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:31.104792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521c24b-3aa9-4364-8f8c-a06d3d8b9bdb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:31.105076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521c24b-3aa9-4364-8f8c-a06d3d8b9bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:31.105353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521c24b-3aa9-4364-8f8c-a06d3d8b9bdb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:31.106513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521c24b-3aa9-4364-8f8c-a06d3d8b9bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:31.107930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521c24b-3aa9-4364-8f8c-a06d3d8b9bdb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:31.109572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 558/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:31.115958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda341-4ab0-48af-a15b-15bd59dc029d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:31.174018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:31.213048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda341-4ab0-48af-a15b-15bd59dc029d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:31.246285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559cb9c-28f8-4c20-baf4-4ba23084cbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:31.246933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559cb9c-28f8-4c20-baf4-4ba23084cbae None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:31.247214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559cb9c-28f8-4c20-baf4-4ba23084cbae None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:31.247488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559cb9c-28f8-4c20-baf4-4ba23084cbae None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:31.248182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559cb9c-28f8-4c20-baf4-4ba23084cbae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:31.249661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559cb9c-28f8-4c20-baf4-4ba23084cbae None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:31.275201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 576/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:22:31 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:22:33.336234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c3478-218f-4812-a469-f322ce338a64 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:33.336787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c3478-218f-4812-a469-f322ce338a64 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:33.336787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c3478-218f-4812-a469-f322ce338a64 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:33.338119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 559/1135] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:33.343321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559c8a0-1024-4781-aacf-f0662c0831a4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:33.343739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559c8a0-1024-4781-aacf-f0662c0831a4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:33.343891 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559c8a0-1024-4781-aacf-f0662c0831a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:33.360900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0559c8a0-1024-4781-aacf-f0662c0831a4 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:33.362079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0559c8a0-1024-4781-aacf-f0662c0831a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:33.426963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 577/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:33.451622 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c3478-218f-4812-a469-f322ce338a64 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:33.531132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:33.550912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c3478-218f-4812-a469-f322ce338a64 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:33.583182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b683138-08f4-462f-8775-45170464e709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:33.584659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683138-08f4-462f-8775-45170464e709 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:33.584937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683138-08f4-462f-8775-45170464e709 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:33.585208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683138-08f4-462f-8775-45170464e709 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:33.586418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b683138-08f4-462f-8775-45170464e709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:33.587775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b683138-08f4-462f-8775-45170464e709 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:33.589384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 560/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:22:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:33.595077 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559c8a0-1024-4781-aacf-f0662c0831a4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:33.640736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:33.693437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559c8a0-1024-4781-aacf-f0662c0831a4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:33.726920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c50053-1977-42d9-b7f3-e9d86fe79d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:33.727586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c50053-1977-42d9-b7f3-e9d86fe79d2a None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:33.727870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c50053-1977-42d9-b7f3-e9d86fe79d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:33.728148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c50053-1977-42d9-b7f3-e9d86fe79d2a None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:33.728874 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c50053-1977-42d9-b7f3-e9d86fe79d2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:33.730307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c50053-1977-42d9-b7f3-e9d86fe79d2a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:33.749014 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 578/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:22:33 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:22:35.788976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2caf-133c-46f1-8ca7-a03e7d15fd1c None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:35.789522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2caf-133c-46f1-8ca7-a03e7d15fd1c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:35.789522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2caf-133c-46f1-8ca7-a03e7d15fd1c None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:35.790869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 561/1139] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:35.795897 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103ea4a-178a-4666-bb04-cf27c316386b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:35.796336 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103ea4a-178a-4666-bb04-cf27c316386b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:35.796591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103ea4a-178a-4666-bb04-cf27c316386b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:35.814156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9103ea4a-178a-4666-bb04-cf27c316386b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:35.815328 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9103ea4a-178a-4666-bb04-cf27c316386b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:35.881984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 579/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:35.905506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2caf-133c-46f1-8ca7-a03e7d15fd1c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:36.005618 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2caf-133c-46f1-8ca7-a03e7d15fd1c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:36.038550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99d986-4924-4ea5-90ae-85a2623098cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:36.039386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99d986-4924-4ea5-90ae-85a2623098cc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:36.039618 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99d986-4924-4ea5-90ae-85a2623098cc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:36.039893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99d986-4924-4ea5-90ae-85a2623098cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:36.049697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99d986-4924-4ea5-90ae-85a2623098cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-dd99d986-4924-4ea5-90ae-85a2623098cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:36.057415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:36.058409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:36.059355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 562/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:22:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:36.064710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103ea4a-178a-4666-bb04-cf27c316386b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:36.094477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:36.162220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103ea4a-178a-4666-bb04-cf27c316386b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:36.194594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91202-75d2-47e2-8fd4-bfa6c6fc9bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:36.195249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91202-75d2-47e2-8fd4-bfa6c6fc9bbd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:36.195530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91202-75d2-47e2-8fd4-bfa6c6fc9bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:36.195803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91202-75d2-47e2-8fd4-bfa6c6fc9bbd None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:36.196565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91202-75d2-47e2-8fd4-bfa6c6fc9bbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:36.196870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91202-75d2-47e2-8fd4-bfa6c6fc9bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:36.198508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91202-75d2-47e2-8fd4-bfa6c6fc9bbd None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:36.206210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 580/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:22:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:36.212561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99d986-4924-4ea5-90ae-85a2623098cc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:36.223472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:36.308901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99d986-4924-4ea5-90ae-85a2623098cc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:36.341778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1dd5b-34b8-4363-943c-7c05dc5d686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:36.342637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1dd5b-34b8-4363-943c-7c05dc5d686c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:36.342918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1dd5b-34b8-4363-943c-7c05dc5d686c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:36.343192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1dd5b-34b8-4363-943c-7c05dc5d686c None admin] PATH_INFO: `/v3/users/e06e1d1b5c954e4bb3aacb5e58b4765d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:36.355995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1dd5b-34b8-4363-943c-7c05dc5d686c None admin] RBAC: Authorizing `identity:get_user(user_id=e06e1d1b5c954e4bb3aacb5e58b4765d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:36.357446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1dd5b-34b8-4363-943c-7c05dc5d686c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:36.359081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 563/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:36 2022] GET /identity/v3/users/e06e1d1b5c954e4bb3aacb5e58b4765d => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:36.365459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91202-75d2-47e2-8fd4-bfa6c6fc9bbd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:36.454065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:36.462866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91202-75d2-47e2-8fd4-bfa6c6fc9bbd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:36.495105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a055dc-9580-48be-82f6-9933599e2443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:36.495740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a055dc-9580-48be-82f6-9933599e2443 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:36.496023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a055dc-9580-48be-82f6-9933599e2443 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:36.496314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a055dc-9580-48be-82f6-9933599e2443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:36.560236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a055dc-9580-48be-82f6-9933599e2443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:36.561668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a055dc-9580-48be-82f6-9933599e2443 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:36.649768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:36.659086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 581/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:22:36.665477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1dd5b-34b8-4363-943c-7c05dc5d686c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:36.728524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:36.763442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1dd5b-34b8-4363-943c-7c05dc5d686c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:36.796233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f83521-ae0e-4e83-b15c-32b1f3f1d956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:36.797102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f83521-ae0e-4e83-b15c-32b1f3f1d956 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:36.797379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f83521-ae0e-4e83-b15c-32b1f3f1d956 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:36.797649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f83521-ae0e-4e83-b15c-32b1f3f1d956 None admin] PATH_INFO: `/v3/projects/f4563585dfd7463f94a167a925ce1174` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:36.798801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f83521-ae0e-4e83-b15c-32b1f3f1d956 None admin] RBAC: Authorizing `identity:get_project(project_id=f4563585dfd7463f94a167a925ce1174)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:36.800201 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f83521-ae0e-4e83-b15c-32b1f3f1d956 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:36.801966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 564/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:36 2022] GET /identity/v3/projects/f4563585dfd7463f94a167a925ce1174 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:36.807978 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a055dc-9580-48be-82f6-9933599e2443 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:36.904716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a055dc-9580-48be-82f6-9933599e2443 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:36.936864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dd7f0-d57c-4b6c-a53b-7363e3ef8c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:36.937516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dd7f0-d57c-4b6c-a53b-7363e3ef8c17 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:36.937796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dd7f0-d57c-4b6c-a53b-7363e3ef8c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:36.938069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dd7f0-d57c-4b6c-a53b-7363e3ef8c17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:36.938891 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dd7f0-d57c-4b6c-a53b-7363e3ef8c17 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=f4563585dfd7463f94a167a925ce1174, user.id=e06e1d1b5c954e4bb3aacb5e58b4765d) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:36.939199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dd7f0-d57c-4b6c-a53b-7363e3ef8c17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:36.940632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dd7f0-d57c-4b6c-a53b-7363e3ef8c17 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:36.946511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 582/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:22:36 2022] GET /identity/v3/role_assignments?user.id=e06e1d1b5c954e4bb3aacb5e58b4765d&scope.project.id=f4563585dfd7463f94a167a925ce1174&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:38.990189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0a373-b91a-4b83-8900-35fa2a3c04e1 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:38.990671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0a373-b91a-4b83-8900-35fa2a3c04e1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:38.990759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0a373-b91a-4b83-8900-35fa2a3c04e1 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:38.992093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 565/1147] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:38.997157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674604f-f160-4a2a-957d-baeeed47e20f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:38.997557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674604f-f160-4a2a-957d-baeeed47e20f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:38.997702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674604f-f160-4a2a-957d-baeeed47e20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:39.014435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9674604f-f160-4a2a-957d-baeeed47e20f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:39.015578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9674604f-f160-4a2a-957d-baeeed47e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:39.064165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:39.081015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 583/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:22:39.104101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0a373-b91a-4b83-8900-35fa2a3c04e1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:39.201110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0a373-b91a-4b83-8900-35fa2a3c04e1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:39.233402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b028d-d80f-4825-b1d2-509cfd432646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:39.234172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b028d-d80f-4825-b1d2-509cfd432646 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:39.234450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b028d-d80f-4825-b1d2-509cfd432646 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:39.234722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b028d-d80f-4825-b1d2-509cfd432646 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:39.236544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:39.245851 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b028d-d80f-4825-b1d2-509cfd432646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-cc8b028d-d80f-4825-b1d2-509cfd432646 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:39.253383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:39.254577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:39.256203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 566/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:22:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:39.261201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674604f-f160-4a2a-957d-baeeed47e20f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:39.355505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674604f-f160-4a2a-957d-baeeed47e20f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:39.389045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8192490-992e-44c0-bc28-5c383c21a093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:39.389701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8192490-992e-44c0-bc28-5c383c21a093 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:39.389977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8192490-992e-44c0-bc28-5c383c21a093 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:39.390245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8192490-992e-44c0-bc28-5c383c21a093 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:39.390981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8192490-992e-44c0-bc28-5c383c21a093 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:39.391282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8192490-992e-44c0-bc28-5c383c21a093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:39.392682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8192490-992e-44c0-bc28-5c383c21a093 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:39.400821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 584/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:22:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:39.406875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b028d-d80f-4825-b1d2-509cfd432646 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:39.504639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b028d-d80f-4825-b1d2-509cfd432646 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:39.536768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4242ff4-b4c1-4ef9-a024-2be0cfa8dbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:39.537556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4242ff4-b4c1-4ef9-a024-2be0cfa8dbfa None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:39.537837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4242ff4-b4c1-4ef9-a024-2be0cfa8dbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:39.538111 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4242ff4-b4c1-4ef9-a024-2be0cfa8dbfa None admin] PATH_INFO: `/v3/users/e06e1d1b5c954e4bb3aacb5e58b4765d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:39.539278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4242ff4-b4c1-4ef9-a024-2be0cfa8dbfa None admin] RBAC: Authorizing `identity:get_user(user_id=e06e1d1b5c954e4bb3aacb5e58b4765d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:39.540721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4242ff4-b4c1-4ef9-a024-2be0cfa8dbfa None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:39.542330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 567/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:39 2022] GET /identity/v3/users/e06e1d1b5c954e4bb3aacb5e58b4765d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:39.547935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8192490-992e-44c0-bc28-5c383c21a093 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:39.595911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:39.645376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8192490-992e-44c0-bc28-5c383c21a093 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:39.678224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848938e-2250-45a3-88dd-35d0d25821ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:39.679056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848938e-2250-45a3-88dd-35d0d25821ce None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:39.679335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848938e-2250-45a3-88dd-35d0d25821ce None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:39.679614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848938e-2250-45a3-88dd-35d0d25821ce None admin] PATH_INFO: `/v3/projects/f4563585dfd7463f94a167a925ce1174` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:39.680780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848938e-2250-45a3-88dd-35d0d25821ce None admin] RBAC: Authorizing `identity:get_project(project_id=f4563585dfd7463f94a167a925ce1174)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:39.682205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848938e-2250-45a3-88dd-35d0d25821ce None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:39.683853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 585/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:39 2022] GET /identity/v3/projects/f4563585dfd7463f94a167a925ce1174 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:39.689802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4242ff4-b4c1-4ef9-a024-2be0cfa8dbfa None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:39.750581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:39.788049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4242ff4-b4c1-4ef9-a024-2be0cfa8dbfa None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:39.821123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab6d76-1ca9-4ccb-b9da-d2f799dca54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:39.821998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab6d76-1ca9-4ccb-b9da-d2f799dca54f None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:39.822533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab6d76-1ca9-4ccb-b9da-d2f799dca54f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:39.822809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab6d76-1ca9-4ccb-b9da-d2f799dca54f None admin] PATH_INFO: `/v3/projects/f4563585dfd7463f94a167a925ce1174/users/e06e1d1b5c954e4bb3aacb5e58b4765d/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:39.824772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab6d76-1ca9-4ccb-b9da-d2f799dca54f None admin] RBAC: Authorizing `identity:create_grant(project_id=f4563585dfd7463f94a167a925ce1174, user_id=e06e1d1b5c954e4bb3aacb5e58b4765d, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:39.826200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab6d76-1ca9-4ccb-b9da-d2f799dca54f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:39.838952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 568/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:22:39 2022] PUT /identity/v3/projects/f4563585dfd7463f94a167a925ce1174/users/e06e1d1b5c954e4bb3aacb5e58b4765d/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:22:41.854201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e03c83-b793-4723-aece-4e3b9f4a99fa None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:41.854730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e03c83-b793-4723-aece-4e3b9f4a99fa None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:41.854730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e03c83-b793-4723-aece-4e3b9f4a99fa None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:41.856092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 586/1154] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:41.861426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d3b7e-e2cc-48e0-b618-7e74c34d88d5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:41.861800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d3b7e-e2cc-48e0-b618-7e74c34d88d5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:41.861975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d3b7e-e2cc-48e0-b618-7e74c34d88d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:41.878266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a08d3b7e-e2cc-48e0-b618-7e74c34d88d5 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:41.879431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a08d3b7e-e2cc-48e0-b618-7e74c34d88d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:41.945100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 569/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:22:41.968773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e03c83-b793-4723-aece-4e3b9f4a99fa None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:42.073128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e03c83-b793-4723-aece-4e3b9f4a99fa None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:42.093406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:42.111291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38fffd-5afa-4d76-83fa-25af147b5c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:42.112024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38fffd-5afa-4d76-83fa-25af147b5c6c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:42.112333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38fffd-5afa-4d76-83fa-25af147b5c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:42.112615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38fffd-5afa-4d76-83fa-25af147b5c6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:42.123403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38fffd-5afa-4d76-83fa-25af147b5c6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-4b38fffd-5afa-4d76-83fa-25af147b5c6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:42.131711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:42.132840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:42.133734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 587/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:22:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:42.138960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d3b7e-e2cc-48e0-b618-7e74c34d88d5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:42.189053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:42.234656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d3b7e-e2cc-48e0-b618-7e74c34d88d5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:42.267375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92735e7-2226-48d8-ab1f-1a5a1d8ef097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:42.268026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92735e7-2226-48d8-ab1f-1a5a1d8ef097 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:42.268323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92735e7-2226-48d8-ab1f-1a5a1d8ef097 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:42.268600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92735e7-2226-48d8-ab1f-1a5a1d8ef097 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:42.269362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92735e7-2226-48d8-ab1f-1a5a1d8ef097 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:42.269663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92735e7-2226-48d8-ab1f-1a5a1d8ef097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:42.271034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92735e7-2226-48d8-ab1f-1a5a1d8ef097 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:42.278727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 570/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:22:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:42.284392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38fffd-5afa-4d76-83fa-25af147b5c6c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:42.380928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38fffd-5afa-4d76-83fa-25af147b5c6c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:42.413230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5028-cd47-4e1f-9683-e3910b8a85a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:42.414079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5028-cd47-4e1f-9683-e3910b8a85a4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:42.414362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5028-cd47-4e1f-9683-e3910b8a85a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:42.414637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5028-cd47-4e1f-9683-e3910b8a85a4 None admin] PATH_INFO: `/v3/users/e06e1d1b5c954e4bb3aacb5e58b4765d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:42.415809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d5028-cd47-4e1f-9683-e3910b8a85a4 None admin] RBAC: Authorizing `identity:get_user(user_id=e06e1d1b5c954e4bb3aacb5e58b4765d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:42.417265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d5028-cd47-4e1f-9683-e3910b8a85a4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:42.418844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 588/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:42 2022] GET /identity/v3/users/e06e1d1b5c954e4bb3aacb5e58b4765d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:42.424482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92735e7-2226-48d8-ab1f-1a5a1d8ef097 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:42.522970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92735e7-2226-48d8-ab1f-1a5a1d8ef097 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:42.535856 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:42.555784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820cccf-b639-459f-bfd7-c142452c96fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:42.556448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820cccf-b639-459f-bfd7-c142452c96fc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:42.556729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820cccf-b639-459f-bfd7-c142452c96fc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:42.557002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820cccf-b639-459f-bfd7-c142452c96fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:42.622013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820cccf-b639-459f-bfd7-c142452c96fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:42.623455 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820cccf-b639-459f-bfd7-c142452c96fc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:42.718582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 571/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:22:42.724877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5028-cd47-4e1f-9683-e3910b8a85a4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:42.769670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:42.822893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5028-cd47-4e1f-9683-e3910b8a85a4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:42.855752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9acdf-514e-4563-b0aa-9e13d4a95b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:42.856621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9acdf-514e-4563-b0aa-9e13d4a95b4e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:42.856906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9acdf-514e-4563-b0aa-9e13d4a95b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:42.857183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9acdf-514e-4563-b0aa-9e13d4a95b4e None admin] PATH_INFO: `/v3/projects/f4563585dfd7463f94a167a925ce1174` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:42.858350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9acdf-514e-4563-b0aa-9e13d4a95b4e None admin] RBAC: Authorizing `identity:get_project(project_id=f4563585dfd7463f94a167a925ce1174)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:42.859782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9acdf-514e-4563-b0aa-9e13d4a95b4e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:42.861485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 589/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:42 2022] GET /identity/v3/projects/f4563585dfd7463f94a167a925ce1174 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:42.867520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820cccf-b639-459f-bfd7-c142452c96fc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:42.882630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:42.966427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820cccf-b639-459f-bfd7-c142452c96fc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:42.999076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db2b73-ed20-4d29-baea-7d8d82d21e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:42.999660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db2b73-ed20-4d29-baea-7d8d82d21e1e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:42.999941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db2b73-ed20-4d29-baea-7d8d82d21e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:43.000214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db2b73-ed20-4d29-baea-7d8d82d21e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:43.001068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db2b73-ed20-4d29-baea-7d8d82d21e1e None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=f4563585dfd7463f94a167a925ce1174, user.id=e06e1d1b5c954e4bb3aacb5e58b4765d) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:43.001380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db2b73-ed20-4d29-baea-7d8d82d21e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:43.002787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db2b73-ed20-4d29-baea-7d8d82d21e1e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:43.009009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 572/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:22:42 2022] GET /identity/v3/role_assignments?user.id=e06e1d1b5c954e4bb3aacb5e58b4765d&scope.project.id=f4563585dfd7463f94a167a925ce1174&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:45.039068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93345b-9c23-43dc-b6c4-87fc809c8894 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:45.039571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93345b-9c23-43dc-b6c4-87fc809c8894 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:45.039657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93345b-9c23-43dc-b6c4-87fc809c8894 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:45.041043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 590/1162] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:45.046173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1eb23-3c11-42c6-9259-1b70a257ec11 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:45.046641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1eb23-3c11-42c6-9259-1b70a257ec11 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:45.046729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1eb23-3c11-42c6-9259-1b70a257ec11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:45.062976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-02f1eb23-3c11-42c6-9259-1b70a257ec11 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:45.064151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-02f1eb23-3c11-42c6-9259-1b70a257ec11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:45.128999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 573/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:45.151986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93345b-9c23-43dc-b6c4-87fc809c8894 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:45.250319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93345b-9c23-43dc-b6c4-87fc809c8894 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:45.263215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:45.283759 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba5360-ee29-4124-8d04-a4af393863fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:45.284416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba5360-ee29-4124-8d04-a4af393863fb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:45.284699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba5360-ee29-4124-8d04-a4af393863fb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:45.284972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba5360-ee29-4124-8d04-a4af393863fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:45.349783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba5360-ee29-4124-8d04-a4af393863fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:45.351254 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba5360-ee29-4124-8d04-a4af393863fb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:45.444746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 591/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:22:45.450916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1eb23-3c11-42c6-9259-1b70a257ec11 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:45.461318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:45.549012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1eb23-3c11-42c6-9259-1b70a257ec11 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:45.582720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16456836-175b-49d7-8036-feec8c2b8f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:45.583506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16456836-175b-49d7-8036-feec8c2b8f3d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:45.583782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16456836-175b-49d7-8036-feec8c2b8f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:45.584057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16456836-175b-49d7-8036-feec8c2b8f3d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:45.590563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16456836-175b-49d7-8036-feec8c2b8f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:45.591996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16456836-175b-49d7-8036-feec8c2b8f3d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-16456836-175b-49d7-8036-feec8c2b8f3d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:45.605761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:45.606957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:22:45.608086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 574/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jul 21 17:22:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:45.612783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba5360-ee29-4124-8d04-a4af393863fb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:45.627338 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:45.710419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba5360-ee29-4124-8d04-a4af393863fb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:45.743765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6b190-1ce6-44e9-befb-e172df39a64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:45.744440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6b190-1ce6-44e9-befb-e172df39a64a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:45.744725 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6b190-1ce6-44e9-befb-e172df39a64a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:45.745004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6b190-1ce6-44e9-befb-e172df39a64a None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:45.745718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6b190-1ce6-44e9-befb-e172df39a64a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:45.746041 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6b190-1ce6-44e9-befb-e172df39a64a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:45.747411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6b190-1ce6-44e9-befb-e172df39a64a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:45.754353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 592/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Jul 21 17:22:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:47.739314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7827f3-042e-4db1-954a-8f47e81a3b35 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:47.739842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7827f3-042e-4db1-954a-8f47e81a3b35 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:47.739842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7827f3-042e-4db1-954a-8f47e81a3b35 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:47.741246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 575/1167] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:47.746604 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0d3e-0542-4894-bff3-fe26681f17cf None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:47.747022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0d3e-0542-4894-bff3-fe26681f17cf None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:47.747160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0d3e-0542-4894-bff3-fe26681f17cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:47.763407 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-555c0d3e-0542-4894-bff3-fe26681f17cf None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:47.764622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-555c0d3e-0542-4894-bff3-fe26681f17cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:47.831273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 593/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:47.855081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7827f3-042e-4db1-954a-8f47e81a3b35 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:47.923171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:47.959406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7827f3-042e-4db1-954a-8f47e81a3b35 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:47.992137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a977190-d1f1-4f9c-900a-084a2776aec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:47.992766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a977190-d1f1-4f9c-900a-084a2776aec1 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:47.993049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a977190-d1f1-4f9c-900a-084a2776aec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:47.993326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a977190-d1f1-4f9c-900a-084a2776aec1 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:47.994069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a977190-d1f1-4f9c-900a-084a2776aec1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:47.995429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a977190-d1f1-4f9c-900a-084a2776aec1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:48.009970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 576/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Thu Jul 21 17:22:47 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:22:50.123109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8e701-bd2a-4f79-a3a2-b960c672e82e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:50.123626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8e701-bd2a-4f79-a3a2-b960c672e82e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:50.123626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8e701-bd2a-4f79-a3a2-b960c672e82e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:50.125051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 594/1170] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:50.130803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbeaff-89ba-4450-9065-ed89870210e9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:50.131223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbeaff-89ba-4450-9065-ed89870210e9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:50.131352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbeaff-89ba-4450-9065-ed89870210e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:50.148010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a2dbeaff-89ba-4450-9065-ed89870210e9 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:50.149285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a2dbeaff-89ba-4450-9065-ed89870210e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:50.213884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 577/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:50.237005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8e701-bd2a-4f79-a3a2-b960c672e82e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:50.246387 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:50.335998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8e701-bd2a-4f79-a3a2-b960c672e82e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:50.368988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b395a3-fbcb-4937-8964-b79f2f49a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:50.369824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b395a3-fbcb-4937-8964-b79f2f49a62d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:50.370103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b395a3-fbcb-4937-8964-b79f2f49a62d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:50.370374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b395a3-fbcb-4937-8964-b79f2f49a62d None admin] PATH_INFO: `/v3/domains/1b673311abff4aafae19028c8d669418` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:50.380470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b395a3-fbcb-4937-8964-b79f2f49a62d None admin] RBAC: Authorizing `identity:get_domain(domain_id=1b673311abff4aafae19028c8d669418)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:50.381901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b395a3-fbcb-4937-8964-b79f2f49a62d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:50.383491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 595/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Thu Jul 21 17:22:50 2022] GET /identity/v3/domains/1b673311abff4aafae19028c8d669418 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:50.389282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbeaff-89ba-4450-9065-ed89870210e9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:50.445642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:50.486437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbeaff-89ba-4450-9065-ed89870210e9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:50.518842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c10a4-05e5-46b1-a4da-af90fef12b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:50.519492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c10a4-05e5-46b1-a4da-af90fef12b47 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:50.520320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c10a4-05e5-46b1-a4da-af90fef12b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:50.520610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c10a4-05e5-46b1-a4da-af90fef12b47 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:50.521320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c10a4-05e5-46b1-a4da-af90fef12b47 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:50.522728 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c10a4-05e5-46b1-a4da-af90fef12b47 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:50.546673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 578/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:22:50 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:22:52.573593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8ee6c-2fb7-4f79-9d21-c7b1679c4c14 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:52.574139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8ee6c-2fb7-4f79-9d21-c7b1679c4c14 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:52.574139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8ee6c-2fb7-4f79-9d21-c7b1679c4c14 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:52.575488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 596/1174] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:52.580756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad42857-d90c-43a9-ac2c-6d2a94cfbd0a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:52.581123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad42857-d90c-43a9-ac2c-6d2a94cfbd0a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:52.581318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad42857-d90c-43a9-ac2c-6d2a94cfbd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:52.599206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-9ad42857-d90c-43a9-ac2c-6d2a94cfbd0a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:52.600408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9ad42857-d90c-43a9-ac2c-6d2a94cfbd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:52.664315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 579/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:52.689058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8ee6c-2fb7-4f79-9d21-c7b1679c4c14 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:52.753599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:52.790782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8ee6c-2fb7-4f79-9d21-c7b1679c4c14 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:52.823009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65980329-f02d-4768-993f-e82d45baaa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:52.823901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65980329-f02d-4768-993f-e82d45baaa4d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:52.824129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65980329-f02d-4768-993f-e82d45baaa4d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:52.824436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65980329-f02d-4768-993f-e82d45baaa4d None admin] PATH_INFO: `/v3/domains/1b673311abff4aafae19028c8d669418` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:52.825586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65980329-f02d-4768-993f-e82d45baaa4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=1b673311abff4aafae19028c8d669418)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:52.827003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65980329-f02d-4768-993f-e82d45baaa4d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:52.828612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 597/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Thu Jul 21 17:22:52 2022] GET /identity/v3/domains/1b673311abff4aafae19028c8d669418 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:52.835336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad42857-d90c-43a9-ac2c-6d2a94cfbd0a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:52.876170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:52.934535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad42857-d90c-43a9-ac2c-6d2a94cfbd0a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:52.968587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81083ad-c5b8-4ade-b51e-18fd06729c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:52.969256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81083ad-c5b8-4ade-b51e-18fd06729c71 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:52.969536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81083ad-c5b8-4ade-b51e-18fd06729c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:52.969809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81083ad-c5b8-4ade-b51e-18fd06729c71 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:52.970507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81083ad-c5b8-4ade-b51e-18fd06729c71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:52.971925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81083ad-c5b8-4ade-b51e-18fd06729c71 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:52.992739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 580/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:22:52 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:22:55.031956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56760bd0-eb06-4be5-83b3-ef91b67493fe None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:55.032592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56760bd0-eb06-4be5-83b3-ef91b67493fe None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:55.032592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56760bd0-eb06-4be5-83b3-ef91b67493fe None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:55.033881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 598/1178] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:55.039121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc96836-af50-4c10-abca-a1bd0da55fec None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:55.039518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc96836-af50-4c10-abca-a1bd0da55fec None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:55.039684 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc96836-af50-4c10-abca-a1bd0da55fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:55.058021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4dc96836-af50-4c10-abca-a1bd0da55fec None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:55.059209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4dc96836-af50-4c10-abca-a1bd0da55fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:55.124481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 581/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:55.147437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56760bd0-eb06-4be5-83b3-ef91b67493fe None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:55.245890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56760bd0-eb06-4be5-83b3-ef91b67493fe None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:55.277809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f28325-013e-4423-a01f-11457c5e6e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:55.278584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f28325-013e-4423-a01f-11457c5e6e5c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:55.278863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f28325-013e-4423-a01f-11457c5e6e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:55.279136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f28325-013e-4423-a01f-11457c5e6e5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:55.280956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:55.290720 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f28325-013e-4423-a01f-11457c5e6e5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b4f28325-013e-4423-a01f-11457c5e6e5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:55.298180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:55.299377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:55.300505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 599/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:22:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:55.305848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc96836-af50-4c10-abca-a1bd0da55fec None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:55.332879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:55.403991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc96836-af50-4c10-abca-a1bd0da55fec None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:55.437125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b5b2b-fb12-44c5-b324-dcd21bd8e2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:55.437781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b5b2b-fb12-44c5-b324-dcd21bd8e2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:55.438063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b5b2b-fb12-44c5-b324-dcd21bd8e2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:55.438338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b5b2b-fb12-44c5-b324-dcd21bd8e2a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:55.439076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b5b2b-fb12-44c5-b324-dcd21bd8e2a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:55.439383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b5b2b-fb12-44c5-b324-dcd21bd8e2a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:55.440759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b5b2b-fb12-44c5-b324-dcd21bd8e2a3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:55.448908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 582/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:22:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:55.454556 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f28325-013e-4423-a01f-11457c5e6e5c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:55.551322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f28325-013e-4423-a01f-11457c5e6e5c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:55.583775 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df527476-d763-49fd-8136-ceebde8da2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:55.584644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df527476-d763-49fd-8136-ceebde8da2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:55.584929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df527476-d763-49fd-8136-ceebde8da2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:55.585202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df527476-d763-49fd-8136-ceebde8da2f4 None admin] PATH_INFO: `/v3/users/473c2ef0df3145469bc4eac87d40bf53` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:55.598225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df527476-d763-49fd-8136-ceebde8da2f4 None admin] RBAC: Authorizing `identity:get_user(user_id=473c2ef0df3145469bc4eac87d40bf53)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:55.599697 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df527476-d763-49fd-8136-ceebde8da2f4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:55.601342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 600/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:55 2022] GET /identity/v3/users/473c2ef0df3145469bc4eac87d40bf53 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:55.606933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b5b2b-fb12-44c5-b324-dcd21bd8e2a3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:55.694044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:55.706115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b5b2b-fb12-44c5-b324-dcd21bd8e2a3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:55.738172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32968321-481a-4b4e-bc74-d918e7619f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:55.739489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32968321-481a-4b4e-bc74-d918e7619f9b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:55.739777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32968321-481a-4b4e-bc74-d918e7619f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:55.740052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32968321-481a-4b4e-bc74-d918e7619f9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:55.804580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32968321-481a-4b4e-bc74-d918e7619f9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:55.805945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32968321-481a-4b4e-bc74-d918e7619f9b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:55.890082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:55.902254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 583/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:22:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:22:55.908070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df527476-d763-49fd-8136-ceebde8da2f4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:55.960000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:56.006143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df527476-d763-49fd-8136-ceebde8da2f4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:56.037950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad107b-1dbe-414d-b263-8a9b3c6b1a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:56.038794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad107b-1dbe-414d-b263-8a9b3c6b1a6d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:56.039075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad107b-1dbe-414d-b263-8a9b3c6b1a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:56.039348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad107b-1dbe-414d-b263-8a9b3c6b1a6d None admin] PATH_INFO: `/v3/projects/d82a5d815c054ef0b38ead22f3810d49` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:56.040524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad107b-1dbe-414d-b263-8a9b3c6b1a6d None admin] RBAC: Authorizing `identity:get_project(project_id=d82a5d815c054ef0b38ead22f3810d49)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:56.041977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad107b-1dbe-414d-b263-8a9b3c6b1a6d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:56.043637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 601/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:55 2022] GET /identity/v3/projects/d82a5d815c054ef0b38ead22f3810d49 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:56.049892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32968321-481a-4b4e-bc74-d918e7619f9b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:56.146659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32968321-481a-4b4e-bc74-d918e7619f9b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:56.179327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4084eea-85fc-4514-a5e3-50de4dc9bd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:56.179964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4084eea-85fc-4514-a5e3-50de4dc9bd60 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:56.180243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4084eea-85fc-4514-a5e3-50de4dc9bd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:56.180540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4084eea-85fc-4514-a5e3-50de4dc9bd60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:56.181362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4084eea-85fc-4514-a5e3-50de4dc9bd60 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=d82a5d815c054ef0b38ead22f3810d49, user.id=473c2ef0df3145469bc4eac87d40bf53) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:56.181665 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4084eea-85fc-4514-a5e3-50de4dc9bd60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:56.183068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4084eea-85fc-4514-a5e3-50de4dc9bd60 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:56.188266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 584/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:22:56 2022] GET /identity/v3/role_assignments?user.id=473c2ef0df3145469bc4eac87d40bf53&scope.project.id=d82a5d815c054ef0b38ead22f3810d49&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:58.163849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77215294-5057-4da3-ba56-2a9abef573dc None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:58.164423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77215294-5057-4da3-ba56-2a9abef573dc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:58.164423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77215294-5057-4da3-ba56-2a9abef573dc None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:58.165779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 602/1186] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:22:58.170998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f854a93-a86d-4491-adac-ae926de1bd47 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:58.171409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f854a93-a86d-4491-adac-ae926de1bd47 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:58.171554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f854a93-a86d-4491-adac-ae926de1bd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:58.189065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5f854a93-a86d-4491-adac-ae926de1bd47 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:22:58.190174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5f854a93-a86d-4491-adac-ae926de1bd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:22:58.243227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:58.263177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 585/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:22:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:22:58.285163 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77215294-5057-4da3-ba56-2a9abef573dc None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:58.384699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77215294-5057-4da3-ba56-2a9abef573dc None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:58.404289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:58.418470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e505c0-cd8e-4cf4-b903-3c698bb0fc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:58.419249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e505c0-cd8e-4cf4-b903-3c698bb0fc71 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:58.419533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e505c0-cd8e-4cf4-b903-3c698bb0fc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:58.419807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e505c0-cd8e-4cf4-b903-3c698bb0fc71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:58.430402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e505c0-cd8e-4cf4-b903-3c698bb0fc71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-e7e505c0-cd8e-4cf4-b903-3c698bb0fc71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:22:58.438049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:22:58.439161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:22:58.440249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 603/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:22:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:22:58.445420 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f854a93-a86d-4491-adac-ae926de1bd47 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:58.541484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f854a93-a86d-4491-adac-ae926de1bd47 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:58.574587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc8e8d-93ad-4f91-a942-8a4b36c6407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:58.575248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc8e8d-93ad-4f91-a942-8a4b36c6407b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:58.575531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc8e8d-93ad-4f91-a942-8a4b36c6407b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:58.575806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc8e8d-93ad-4f91-a942-8a4b36c6407b None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:58.576570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc8e8d-93ad-4f91-a942-8a4b36c6407b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:22:58.576885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc8e8d-93ad-4f91-a942-8a4b36c6407b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:58.578250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc8e8d-93ad-4f91-a942-8a4b36c6407b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:58.586276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 586/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:22:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:58.592148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e505c0-cd8e-4cf4-b903-3c698bb0fc71 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:58.689000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e505c0-cd8e-4cf4-b903-3c698bb0fc71 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:58.720676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5b5d6-6ee9-4318-8630-e8d8f0a0b696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:58.721464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5b5d6-6ee9-4318-8630-e8d8f0a0b696 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:58.721747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5b5d6-6ee9-4318-8630-e8d8f0a0b696 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:58.722017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5b5d6-6ee9-4318-8630-e8d8f0a0b696 None admin] PATH_INFO: `/v3/users/473c2ef0df3145469bc4eac87d40bf53` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:58.723905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5b5d6-6ee9-4318-8630-e8d8f0a0b696 None admin] RBAC: Authorizing `identity:get_user(user_id=473c2ef0df3145469bc4eac87d40bf53)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:58.725341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5b5d6-6ee9-4318-8630-e8d8f0a0b696 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:58.726926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 604/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:22:58 2022] GET /identity/v3/users/473c2ef0df3145469bc4eac87d40bf53 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:22:58.733211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc8e8d-93ad-4f91-a942-8a4b36c6407b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:58.784709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:58.844919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc8e8d-93ad-4f91-a942-8a4b36c6407b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:58.878229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd16387-abb6-4511-b1d3-aa60a72fa274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:58.879098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd16387-abb6-4511-b1d3-aa60a72fa274 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:58.879467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd16387-abb6-4511-b1d3-aa60a72fa274 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:58.879753 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd16387-abb6-4511-b1d3-aa60a72fa274 None admin] PATH_INFO: `/v3/projects/d82a5d815c054ef0b38ead22f3810d49` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:58.880941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd16387-abb6-4511-b1d3-aa60a72fa274 None admin] RBAC: Authorizing `identity:get_project(project_id=d82a5d815c054ef0b38ead22f3810d49)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:58.882438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd16387-abb6-4511-b1d3-aa60a72fa274 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:58.884215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 587/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:22:58 2022] GET /identity/v3/projects/d82a5d815c054ef0b38ead22f3810d49 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:22:58.890304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5b5d6-6ee9-4318-8630-e8d8f0a0b696 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:22:58.938847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:22:58.988948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5b5d6-6ee9-4318-8630-e8d8f0a0b696 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:22:59.021883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3a9db-c26e-4016-a9dd-c6a4f1e84b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:22:59.022763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a9db-c26e-4016-a9dd-c6a4f1e84b76 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:22:59.023045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a9db-c26e-4016-a9dd-c6a4f1e84b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:22:59.023322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a9db-c26e-4016-a9dd-c6a4f1e84b76 None admin] PATH_INFO: `/v3/projects/d82a5d815c054ef0b38ead22f3810d49/users/473c2ef0df3145469bc4eac87d40bf53/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:22:59.025264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3a9db-c26e-4016-a9dd-c6a4f1e84b76 None admin] RBAC: Authorizing `identity:create_grant(project_id=d82a5d815c054ef0b38ead22f3810d49, user_id=473c2ef0df3145469bc4eac87d40bf53, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:22:59.026696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3a9db-c26e-4016-a9dd-c6a4f1e84b76 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:22:59.040075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 605/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:22:58 2022] PUT /identity/v3/projects/d82a5d815c054ef0b38ead22f3810d49/users/473c2ef0df3145469bc4eac87d40bf53/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:23:01.053688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb198f-317e-480b-b7c3-9ec58369b501 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:01.054236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb198f-317e-480b-b7c3-9ec58369b501 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:01.054236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb198f-317e-480b-b7c3-9ec58369b501 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:01.055608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 588/1193] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:23:01.060813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97d9e7-86ff-41e4-a55e-98b1f96c5950 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:01.061204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97d9e7-86ff-41e4-a55e-98b1f96c5950 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:01.061377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97d9e7-86ff-41e4-a55e-98b1f96c5950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:01.078856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ff97d9e7-86ff-41e4-a55e-98b1f96c5950 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:23:01.080030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ff97d9e7-86ff-41e4-a55e-98b1f96c5950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:23:01.150506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 606/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:23:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:23:01.173428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb198f-317e-480b-b7c3-9ec58369b501 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:23:01.275817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb198f-317e-480b-b7c3-9ec58369b501 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:23:01.288408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:23:01.309717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab887b-9cfc-4b59-829a-59236ff16845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:23:01.310501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab887b-9cfc-4b59-829a-59236ff16845 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:01.310782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab887b-9cfc-4b59-829a-59236ff16845 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:01.311058 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab887b-9cfc-4b59-829a-59236ff16845 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:01.322686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab887b-9cfc-4b59-829a-59236ff16845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f3ab887b-9cfc-4b59-829a-59236ff16845 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:23:01.330292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:23:01.331385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:23:01.332569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 589/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:23:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:23:01.337647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97d9e7-86ff-41e4-a55e-98b1f96c5950 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:23:01.375865 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:23:01.435338 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97d9e7-86ff-41e4-a55e-98b1f96c5950 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:23:01.467796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8503b1-bdbe-4b7a-a799-700033360384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:23:01.468472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8503b1-bdbe-4b7a-a799-700033360384 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:01.468754 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8503b1-bdbe-4b7a-a799-700033360384 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:01.469029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8503b1-bdbe-4b7a-a799-700033360384 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:01.469767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8503b1-bdbe-4b7a-a799-700033360384 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:23:01.470073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8503b1-bdbe-4b7a-a799-700033360384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:23:01.471434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8503b1-bdbe-4b7a-a799-700033360384 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:23:01.479990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 607/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:23:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:23:01.485718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab887b-9cfc-4b59-829a-59236ff16845 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:23:01.584501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab887b-9cfc-4b59-829a-59236ff16845 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:23:01.617009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c5b1f-93b5-4642-a531-4984209d25b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:23:01.618013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c5b1f-93b5-4642-a531-4984209d25b5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:01.618291 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c5b1f-93b5-4642-a531-4984209d25b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:01.618561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c5b1f-93b5-4642-a531-4984209d25b5 None admin] PATH_INFO: `/v3/users/473c2ef0df3145469bc4eac87d40bf53` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:01.619730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c5b1f-93b5-4642-a531-4984209d25b5 None admin] RBAC: Authorizing `identity:get_user(user_id=473c2ef0df3145469bc4eac87d40bf53)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:23:01.621167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c5b1f-93b5-4642-a531-4984209d25b5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:23:01.622764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 590/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Jul 21 17:23:01 2022] GET /identity/v3/users/473c2ef0df3145469bc4eac87d40bf53 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:23:01.629108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8503b1-bdbe-4b7a-a799-700033360384 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:23:01.726353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:23:01.729390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8503b1-bdbe-4b7a-a799-700033360384 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:23:01.762584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84608c0-3632-49fb-9389-ba18e5a7a1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:23:01.763212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84608c0-3632-49fb-9389-ba18e5a7a1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:01.763492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84608c0-3632-49fb-9389-ba18e5a7a1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:01.763764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84608c0-3632-49fb-9389-ba18e5a7a1e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:01.828711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84608c0-3632-49fb-9389-ba18e5a7a1e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:23:01.830156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84608c0-3632-49fb-9389-ba18e5a7a1e6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:23:01.921775 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:23:01.926577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 608/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:23:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:23:01.932099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c5b1f-93b5-4642-a531-4984209d25b5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:23:01.972638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:23:02.028077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c5b1f-93b5-4642-a531-4984209d25b5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:23:02.062764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4efa0-097a-4b9e-8350-7bfdb20bc6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:23:02.063605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4efa0-097a-4b9e-8350-7bfdb20bc6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:02.063885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4efa0-097a-4b9e-8350-7bfdb20bc6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:02.064155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4efa0-097a-4b9e-8350-7bfdb20bc6d8 None admin] PATH_INFO: `/v3/projects/d82a5d815c054ef0b38ead22f3810d49` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:02.065358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4efa0-097a-4b9e-8350-7bfdb20bc6d8 None admin] RBAC: Authorizing `identity:get_project(project_id=d82a5d815c054ef0b38ead22f3810d49)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:23:02.067359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4efa0-097a-4b9e-8350-7bfdb20bc6d8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:23:02.069054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 591/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Jul 21 17:23:01 2022] GET /identity/v3/projects/d82a5d815c054ef0b38ead22f3810d49 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:23:02.075368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84608c0-3632-49fb-9389-ba18e5a7a1e6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:23:02.172752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84608c0-3632-49fb-9389-ba18e5a7a1e6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:23:02.205724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f303809-bbc2-409f-8a7b-c15e74095839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:23:02.206361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f303809-bbc2-409f-8a7b-c15e74095839 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:02.206669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f303809-bbc2-409f-8a7b-c15e74095839 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:02.207224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f303809-bbc2-409f-8a7b-c15e74095839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:02.207845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f303809-bbc2-409f-8a7b-c15e74095839 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=d82a5d815c054ef0b38ead22f3810d49, user.id=473c2ef0df3145469bc4eac87d40bf53) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:23:02.208153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f303809-bbc2-409f-8a7b-c15e74095839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:23:02.209595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f303809-bbc2-409f-8a7b-c15e74095839 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:23:02.215186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 609/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:23:02 2022] GET /identity/v3/role_assignments?user.id=473c2ef0df3145469bc4eac87d40bf53&scope.project.id=d82a5d815c054ef0b38ead22f3810d49&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:23:12.058784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=46 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:23:12.212733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=68 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:23:57.464165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280a160-ab16-4665-b897-689dd8c910ca None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:57.464803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280a160-ab16-4665-b897-689dd8c910ca None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:57.464803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280a160-ab16-4665-b897-689dd8c910ca None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:57.466118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 592/1201] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:23:57.474721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10227138-42c7-48e2-ab74-0c2dddba1289 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:57.474721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10227138-42c7-48e2-ab74-0c2dddba1289 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:57.474721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10227138-42c7-48e2-ab74-0c2dddba1289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:57.492980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-10227138-42c7-48e2-ab74-0c2dddba1289 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:23:57.494168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-10227138-42c7-48e2-ab74-0c2dddba1289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:23:57.586992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 610/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:23:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:23:57.612223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280a160-ab16-4665-b897-689dd8c910ca None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:23:57.757013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280a160-ab16-4665-b897-689dd8c910ca None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:23:57.802086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9357d-dcad-4562-a56e-4d126bba69eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:23:57.802898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9357d-dcad-4562-a56e-4d126bba69eb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:57.803181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9357d-dcad-4562-a56e-4d126bba69eb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:57.803461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9357d-dcad-4562-a56e-4d126bba69eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:57.815467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9357d-dcad-4562-a56e-4d126bba69eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:23:57.817032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9357d-dcad-4562-a56e-4d126bba69eb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a4c9357d-dcad-4562-a56e-4d126bba69eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:23:57.839419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:23:57.841112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:23:57.842247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:23:57.842247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:23:57.842247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:23:57.842247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:23:57.845640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 593/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:23:57.847163 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10227138-42c7-48e2-ab74-0c2dddba1289 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:23:58.001114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10227138-42c7-48e2-ab74-0c2dddba1289 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:23:58.039079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:23:58.046342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b09317-3353-4f58-8929-0de02fa6da80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:23:58.048647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b09317-3353-4f58-8929-0de02fa6da80 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:58.048994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b09317-3353-4f58-8929-0de02fa6da80 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:58.049316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b09317-3353-4f58-8929-0de02fa6da80 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:58.050248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b09317-3353-4f58-8929-0de02fa6da80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:23:58.050648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b09317-3353-4f58-8929-0de02fa6da80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:23:58.052076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b09317-3353-4f58-8929-0de02fa6da80 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:23:58.068397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 611/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:23:58.077411 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9357d-dcad-4562-a56e-4d126bba69eb None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:23:58.151462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:23:58.239627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9357d-dcad-4562-a56e-4d126bba69eb None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:23:58.280595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447787b-68c2-463e-9488-1d7029c86653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:23:58.281885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447787b-68c2-463e-9488-1d7029c86653 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:23:58.282196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447787b-68c2-463e-9488-1d7029c86653 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:23:58.282474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447787b-68c2-463e-9488-1d7029c86653 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:23:58.283183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447787b-68c2-463e-9488-1d7029c86653 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:23:58.284662 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447787b-68c2-463e-9488-1d7029c86653 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:23:58.307428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 594/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Jul 21 17:23:58 2022] POST /identity/v3/users => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:24:00.720565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15d9ab-c174-45bf-9368-ef073e32a45d None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:00.721138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15d9ab-c174-45bf-9368-ef073e32a45d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:00.721138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15d9ab-c174-45bf-9368-ef073e32a45d None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:00.722938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 612/1206] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:24:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:24:00.728319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf98b79-7781-4f30-a389-17ace55b4329 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:00.728756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf98b79-7781-4f30-a389-17ace55b4329 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:00.728934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf98b79-7781-4f30-a389-17ace55b4329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:00.747448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3bf98b79-7781-4f30-a389-17ace55b4329 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:24:00.749048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3bf98b79-7781-4f30-a389-17ace55b4329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:24:00.837974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 595/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:24:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:24:00.862323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15d9ab-c174-45bf-9368-ef073e32a45d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:00.990576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15d9ab-c174-45bf-9368-ef073e32a45d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:01.028732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23fe0f-3689-467f-b4c7-d739dd31b7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:01.029938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23fe0f-3689-467f-b4c7-d739dd31b7fe None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:01.030448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23fe0f-3689-467f-b4c7-d739dd31b7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:01.031716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23fe0f-3689-467f-b4c7-d739dd31b7fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:01.046004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23fe0f-3689-467f-b4c7-d739dd31b7fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ed23fe0f-3689-467f-b4c7-d739dd31b7fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:24:01.054934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:01.056109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:24:01.057184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:01.059305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 613/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:24:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:01.066163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf98b79-7781-4f30-a389-17ace55b4329 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:01.113436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:01.183843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf98b79-7781-4f30-a389-17ace55b4329 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:01.225614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221c0eb-81a5-492a-909f-55108d74e434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:01.226349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221c0eb-81a5-492a-909f-55108d74e434 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:01.226632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221c0eb-81a5-492a-909f-55108d74e434 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:01.227032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221c0eb-81a5-492a-909f-55108d74e434 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:01.227805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221c0eb-81a5-492a-909f-55108d74e434 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:01.228114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221c0eb-81a5-492a-909f-55108d74e434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:01.229531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221c0eb-81a5-492a-909f-55108d74e434 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:01.238212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 596/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:24:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:01.244514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23fe0f-3689-467f-b4c7-d739dd31b7fe None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:01.310688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:01.367970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23fe0f-3689-467f-b4c7-d739dd31b7fe None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:01.408683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae759a9-b71e-40eb-8044-bbdad2bfca0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:01.409614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae759a9-b71e-40eb-8044-bbdad2bfca0e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:01.410066 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae759a9-b71e-40eb-8044-bbdad2bfca0e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:01.413040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae759a9-b71e-40eb-8044-bbdad2bfca0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:01.424617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae759a9-b71e-40eb-8044-bbdad2bfca0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:01.426248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae759a9-b71e-40eb-8044-bbdad2bfca0e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-0ae759a9-b71e-40eb-8044-bbdad2bfca0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:01.449176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:01.450644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:01.451608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:01.451608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:01.451608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:01.451608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:01.451608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 614/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:01.457477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221c0eb-81a5-492a-909f-55108d74e434 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:01.569122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:01.573281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221c0eb-81a5-492a-909f-55108d74e434 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:01.614143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe4dd-10c2-47e3-8fa6-2785e87c3846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:01.614879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe4dd-10c2-47e3-8fa6-2785e87c3846 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:01.615195 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe4dd-10c2-47e3-8fa6-2785e87c3846 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:01.615483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe4dd-10c2-47e3-8fa6-2785e87c3846 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:01.616233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbe4dd-10c2-47e3-8fa6-2785e87c3846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:01.616587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbe4dd-10c2-47e3-8fa6-2785e87c3846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:01.618063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbe4dd-10c2-47e3-8fa6-2785e87c3846 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:01.632332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 597/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:01.639297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae759a9-b71e-40eb-8044-bbdad2bfca0e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:01.764595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae759a9-b71e-40eb-8044-bbdad2bfca0e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:01.766761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:01.806234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae8248-25af-453b-81d7-f2d6aeb547ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:01.807198 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae8248-25af-453b-81d7-f2d6aeb547ba None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:01.807468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae8248-25af-453b-81d7-f2d6aeb547ba None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:01.807932 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae8248-25af-453b-81d7-f2d6aeb547ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:01.817822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae8248-25af-453b-81d7-f2d6aeb547ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:01.819624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae8248-25af-453b-81d7-f2d6aeb547ba None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b0ae8248-25af-453b-81d7-f2d6aeb547ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:24:01.836431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:01.837614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:01.838613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 615/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:01.844909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe4dd-10c2-47e3-8fa6-2785e87c3846 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:01.974796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe4dd-10c2-47e3-8fa6-2785e87c3846 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:02.015881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd24888-cc9c-4e38-8e27-b581ec3bd74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:02.016552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd24888-cc9c-4e38-8e27-b581ec3bd74d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:02.016838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd24888-cc9c-4e38-8e27-b581ec3bd74d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:02.018142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd24888-cc9c-4e38-8e27-b581ec3bd74d None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:02.018142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd24888-cc9c-4e38-8e27-b581ec3bd74d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:02.018344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd24888-cc9c-4e38-8e27-b581ec3bd74d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:02.019705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd24888-cc9c-4e38-8e27-b581ec3bd74d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:02.039508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 598/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jul 21 17:24:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:24:02.046207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae8248-25af-453b-81d7-f2d6aeb547ba None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:02.175482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae8248-25af-453b-81d7-f2d6aeb547ba None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:02.222893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb50f0-1d46-4446-a2d6-6ba9070f097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:02.224254 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb50f0-1d46-4446-a2d6-6ba9070f097b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:02.224676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb50f0-1d46-4446-a2d6-6ba9070f097b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:02.224951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb50f0-1d46-4446-a2d6-6ba9070f097b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:02.264683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:02.311893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb50f0-1d46-4446-a2d6-6ba9070f097b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:02.313819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb50f0-1d46-4446-a2d6-6ba9070f097b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:02.454824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 616/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:24:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:24:02.456345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd24888-cc9c-4e38-8e27-b581ec3bd74d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:02.479078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:02.564876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd24888-cc9c-4e38-8e27-b581ec3bd74d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:02.601187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf081ff3-6d00-4696-85b6-387df0b436e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:02.601999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf081ff3-6d00-4696-85b6-387df0b436e8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:02.602290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf081ff3-6d00-4696-85b6-387df0b436e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:02.602588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf081ff3-6d00-4696-85b6-387df0b436e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:02.613339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf081ff3-6d00-4696-85b6-387df0b436e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:02.614779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf081ff3-6d00-4696-85b6-387df0b436e8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-cf081ff3-6d00-4696-85b6-387df0b436e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:02.635447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:02.637435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:02.639905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:02.639905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:02.639905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:02.639905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:02.643058 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 599/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:02.650038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb50f0-1d46-4446-a2d6-6ba9070f097b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:02.735809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:02.785468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb50f0-1d46-4446-a2d6-6ba9070f097b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:02.829715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8cb28-7d7f-4034-b5f2-15d7d9c6aec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:02.830630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8cb28-7d7f-4034-b5f2-15d7d9c6aec4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:02.831071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8cb28-7d7f-4034-b5f2-15d7d9c6aec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:02.831375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8cb28-7d7f-4034-b5f2-15d7d9c6aec4 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:02.832157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8cb28-7d7f-4034-b5f2-15d7d9c6aec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:02.832647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8cb28-7d7f-4034-b5f2-15d7d9c6aec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:02.834709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8cb28-7d7f-4034-b5f2-15d7d9c6aec4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:02.855300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 617/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:02.857183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf081ff3-6d00-4696-85b6-387df0b436e8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:02.949819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:02.985939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf081ff3-6d00-4696-85b6-387df0b436e8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:03.033087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe294d-2d19-4214-8448-faae49830047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:03.034050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe294d-2d19-4214-8448-faae49830047 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:03.034663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe294d-2d19-4214-8448-faae49830047 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:03.034955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe294d-2d19-4214-8448-faae49830047 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:03.043312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe294d-2d19-4214-8448-faae49830047 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:03.045199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe294d-2d19-4214-8448-faae49830047 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-4afe294d-2d19-4214-8448-faae49830047 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:24:03.063599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:03.064600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:03.065501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:24:03.065501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:24:03.065501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:24:03.065501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:24:03.065501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:24:03.065501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:24:03.065501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:03.065501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:03.073351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 600/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:24:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:24:03.074613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8cb28-7d7f-4034-b5f2-15d7d9c6aec4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:03.222634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8cb28-7d7f-4034-b5f2-15d7d9c6aec4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:03.251654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:03.266778 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f52da-9ee9-451a-b486-a8a9e93c1ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:03.267936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f52da-9ee9-451a-b486-a8a9e93c1ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:03.268322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f52da-9ee9-451a-b486-a8a9e93c1ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:03.268684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f52da-9ee9-451a-b486-a8a9e93c1ac8 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:03.269709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f52da-9ee9-451a-b486-a8a9e93c1ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:03.270028 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f52da-9ee9-451a-b486-a8a9e93c1ac8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:03.271716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f52da-9ee9-451a-b486-a8a9e93c1ac8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:03.286320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 618/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:24:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:03.294622 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe294d-2d19-4214-8448-faae49830047 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:03.420323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe294d-2d19-4214-8448-faae49830047 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:03.455990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:03.472945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19242be4-2710-4b3a-9985-785854892b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:03.473615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19242be4-2710-4b3a-9985-785854892b9a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:03.473919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19242be4-2710-4b3a-9985-785854892b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:03.474173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19242be4-2710-4b3a-9985-785854892b9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:03.475004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19242be4-2710-4b3a-9985-785854892b9a None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=901cf70064c34b63a0eda979a5ce77e4) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:03.475317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19242be4-2710-4b3a-9985-785854892b9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:03.476777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19242be4-2710-4b3a-9985-785854892b9a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:03.485291 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 601/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:24:03 2022] GET /identity/v3/role_assignments?user.id=901cf70064c34b63a0eda979a5ce77e4&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:05.673842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a053ef-c912-4ec9-86d4-76e3a421fa52 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:05.674402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a053ef-c912-4ec9-86d4-76e3a421fa52 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:05.674402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a053ef-c912-4ec9-86d4-76e3a421fa52 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:05.675742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 619/1220] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:24:05.681042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ef772-ef3d-45c0-81d6-b0354cd7b242 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:05.681450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ef772-ef3d-45c0-81d6-b0354cd7b242 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:05.681606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ef772-ef3d-45c0-81d6-b0354cd7b242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:05.699742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-702ef772-ef3d-45c0-81d6-b0354cd7b242 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:24:05.701240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-702ef772-ef3d-45c0-81d6-b0354cd7b242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:24:05.775849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 602/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:24:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:24:05.799569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a053ef-c912-4ec9-86d4-76e3a421fa52 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:05.919700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a053ef-c912-4ec9-86d4-76e3a421fa52 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:05.957191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9def49d-0db4-4fcf-ab38-22ee199be905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:05.958109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9def49d-0db4-4fcf-ab38-22ee199be905 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:05.958391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9def49d-0db4-4fcf-ab38-22ee199be905 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:05.958668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9def49d-0db4-4fcf-ab38-22ee199be905 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:05.970301 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9def49d-0db4-4fcf-ab38-22ee199be905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f9def49d-0db4-4fcf-ab38-22ee199be905 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:24:05.978870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:05.979862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:05.981331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 620/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:24:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:05.987455 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ef772-ef3d-45c0-81d6-b0354cd7b242 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:06.092446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ef772-ef3d-45c0-81d6-b0354cd7b242 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:06.118065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:06.129973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f91ca3-1364-4646-8f7c-353bd9cec245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:06.130650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f91ca3-1364-4646-8f7c-353bd9cec245 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:06.130935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f91ca3-1364-4646-8f7c-353bd9cec245 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:06.131216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f91ca3-1364-4646-8f7c-353bd9cec245 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:06.131967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f91ca3-1364-4646-8f7c-353bd9cec245 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:06.132276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f91ca3-1364-4646-8f7c-353bd9cec245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:06.133669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f91ca3-1364-4646-8f7c-353bd9cec245 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:06.141941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 603/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:24:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:24:06.148275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9def49d-0db4-4fcf-ab38-22ee199be905 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:06.201324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:06.264136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9def49d-0db4-4fcf-ab38-22ee199be905 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:06.300473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62bbb4-2a7f-477c-a8da-75d8dec3ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:06.301297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62bbb4-2a7f-477c-a8da-75d8dec3ad48 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:06.301583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62bbb4-2a7f-477c-a8da-75d8dec3ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:06.301883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62bbb4-2a7f-477c-a8da-75d8dec3ad48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:06.313211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62bbb4-2a7f-477c-a8da-75d8dec3ad48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:06.314664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62bbb4-2a7f-477c-a8da-75d8dec3ad48 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-3c62bbb4-2a7f-477c-a8da-75d8dec3ad48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:06.333048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:06.334141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:06.335166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:06.335166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:06.335166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:06.335166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:06.335166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 621/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:06.340421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f91ca3-1364-4646-8f7c-353bd9cec245 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:06.445114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f91ca3-1364-4646-8f7c-353bd9cec245 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:06.481202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975be5b-d84d-4f63-b577-770bbe1ef309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:06.481798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975be5b-d84d-4f63-b577-770bbe1ef309 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:06.482080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975be5b-d84d-4f63-b577-770bbe1ef309 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:06.482356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975be5b-d84d-4f63-b577-770bbe1ef309 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:06.483075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975be5b-d84d-4f63-b577-770bbe1ef309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:06.483382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975be5b-d84d-4f63-b577-770bbe1ef309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:06.484776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975be5b-d84d-4f63-b577-770bbe1ef309 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:06.496365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 604/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:06.502342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62bbb4-2a7f-477c-a8da-75d8dec3ad48 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:06.598930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:06.606130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62bbb4-2a7f-477c-a8da-75d8dec3ad48 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:06.648160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85853216-b8dd-4e49-aa4b-ba4bcdef5ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:06.649313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853216-b8dd-4e49-aa4b-ba4bcdef5ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:06.649786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853216-b8dd-4e49-aa4b-ba4bcdef5ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:06.650271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853216-b8dd-4e49-aa4b-ba4bcdef5ce4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:06.662648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85853216-b8dd-4e49-aa4b-ba4bcdef5ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:06.664287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85853216-b8dd-4e49-aa4b-ba4bcdef5ce4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-85853216-b8dd-4e49-aa4b-ba4bcdef5ce4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:24:06.684209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:06.685843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:06.687221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 622/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:24:06.691839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975be5b-d84d-4f63-b577-770bbe1ef309 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:06.741917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:06.804578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975be5b-d84d-4f63-b577-770bbe1ef309 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:06.848360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79f769-b37a-44ad-ac81-f79bc99eb8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:06.849019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79f769-b37a-44ad-ac81-f79bc99eb8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:06.849295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79f769-b37a-44ad-ac81-f79bc99eb8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:06.849563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79f769-b37a-44ad-ac81-f79bc99eb8b0 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:06.850407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79f769-b37a-44ad-ac81-f79bc99eb8b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:06.850713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79f769-b37a-44ad-ac81-f79bc99eb8b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:06.852130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79f769-b37a-44ad-ac81-f79bc99eb8b0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:06.877647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 605/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jul 21 17:24:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:06.885282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85853216-b8dd-4e49-aa4b-ba4bcdef5ce4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:07.013390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85853216-b8dd-4e49-aa4b-ba4bcdef5ce4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:07.050307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f54fb5-e9fa-48d4-b157-ebae5330bf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:07.051098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f54fb5-e9fa-48d4-b157-ebae5330bf7d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:07.051378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f54fb5-e9fa-48d4-b157-ebae5330bf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:07.051655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f54fb5-e9fa-48d4-b157-ebae5330bf7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:07.063553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f54fb5-e9fa-48d4-b157-ebae5330bf7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:07.065760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f54fb5-e9fa-48d4-b157-ebae5330bf7d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-94f54fb5-e9fa-48d4-b157-ebae5330bf7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:07.090327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:07.091291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:07.092205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:07.092205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:07.092205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:07.092205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:07.092700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 623/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:07.098751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79f769-b37a-44ad-ac81-f79bc99eb8b0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:07.215604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79f769-b37a-44ad-ac81-f79bc99eb8b0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:07.219355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:07.257132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c25d-8224-44e5-a32d-e020304c4041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:07.257781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c25d-8224-44e5-a32d-e020304c4041 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:07.258066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c25d-8224-44e5-a32d-e020304c4041 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:07.258342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c25d-8224-44e5-a32d-e020304c4041 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:07.259060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c25d-8224-44e5-a32d-e020304c4041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:07.259368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c25d-8224-44e5-a32d-e020304c4041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:07.260829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c25d-8224-44e5-a32d-e020304c4041 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:07.274484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 606/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:07.281669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f54fb5-e9fa-48d4-b157-ebae5330bf7d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:07.292615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:07.416110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f54fb5-e9fa-48d4-b157-ebae5330bf7d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:07.455855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff3ecd-9c40-4f0e-84b6-54ba5652c2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:07.457131 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff3ecd-9c40-4f0e-84b6-54ba5652c2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:07.457508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff3ecd-9c40-4f0e-84b6-54ba5652c2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:07.457939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff3ecd-9c40-4f0e-84b6-54ba5652c2b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:07.465255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff3ecd-9c40-4f0e-84b6-54ba5652c2b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:07.466701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff3ecd-9c40-4f0e-84b6-54ba5652c2b6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-c1ff3ecd-9c40-4f0e-84b6-54ba5652c2b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:24:07.480770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:07.481872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:07.483031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:24:07.483031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:24:07.483031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:24:07.483031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:24:07.483031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:24:07.483031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:24:07.483031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:07.483031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:07.483031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 624/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:24:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:07.488145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c25d-8224-44e5-a32d-e020304c4041 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:07.643872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c25d-8224-44e5-a32d-e020304c4041 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:07.688049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105606e-afcf-4e9f-bd78-5c36988176b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:07.688800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105606e-afcf-4e9f-bd78-5c36988176b5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:07.689277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105606e-afcf-4e9f-bd78-5c36988176b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:07.689575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105606e-afcf-4e9f-bd78-5c36988176b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:07.690395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105606e-afcf-4e9f-bd78-5c36988176b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:07.690701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105606e-afcf-4e9f-bd78-5c36988176b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:07.692276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105606e-afcf-4e9f-bd78-5c36988176b5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:07.712817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 607/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:24:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:07.714139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff3ecd-9c40-4f0e-84b6-54ba5652c2b6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:07.791802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:07.832774 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff3ecd-9c40-4f0e-84b6-54ba5652c2b6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:07.875758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82161adc-5230-480f-a70d-8e49ece27ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:07.876792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82161adc-5230-480f-a70d-8e49ece27ba2 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:07.877206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82161adc-5230-480f-a70d-8e49ece27ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:07.877611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82161adc-5230-480f-a70d-8e49ece27ba2 None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/901cf70064c34b63a0eda979a5ce77e4/roles/b9347810c7484efd95599fb9c51ab2df` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:07.894344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82161adc-5230-480f-a70d-8e49ece27ba2 None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=901cf70064c34b63a0eda979a5ce77e4, role_id=b9347810c7484efd95599fb9c51ab2df)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:07.895762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82161adc-5230-480f-a70d-8e49ece27ba2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:07.908630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 625/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:24:07 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/901cf70064c34b63a0eda979a5ce77e4/roles/b9347810c7484efd95599fb9c51ab2df => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:24:10.133874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9a017-8984-49da-aadd-8eaf11d5b5ca None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:10.134445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9a017-8984-49da-aadd-8eaf11d5b5ca None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:10.134445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9a017-8984-49da-aadd-8eaf11d5b5ca None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:10.135801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 608/1233] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:24:10.141995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbda166-9d93-4b15-b0b8-d158e8b0778e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:10.142421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbda166-9d93-4b15-b0b8-d158e8b0778e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:10.142554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbda166-9d93-4b15-b0b8-d158e8b0778e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:10.159472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4bbda166-9d93-4b15-b0b8-d158e8b0778e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:24:10.160692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4bbda166-9d93-4b15-b0b8-d158e8b0778e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:24:10.235821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 626/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:24:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:24:10.261444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9a017-8984-49da-aadd-8eaf11d5b5ca None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:10.289690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:10.379631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9a017-8984-49da-aadd-8eaf11d5b5ca None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:10.419689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a717dd-47aa-4233-a405-b74f5e00902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:10.420813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a717dd-47aa-4233-a405-b74f5e00902d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:10.421180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a717dd-47aa-4233-a405-b74f5e00902d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:10.421460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a717dd-47aa-4233-a405-b74f5e00902d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:10.432279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a717dd-47aa-4233-a405-b74f5e00902d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-37a717dd-47aa-4233-a405-b74f5e00902d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:24:10.440602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:10.441571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:10.442922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 609/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Jul 21 17:24:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:10.448467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbda166-9d93-4b15-b0b8-d158e8b0778e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:10.499607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:10.559305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbda166-9d93-4b15-b0b8-d158e8b0778e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:10.595243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1980bb-fefe-4b64-ab0b-81f33b494790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:10.595919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1980bb-fefe-4b64-ab0b-81f33b494790 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:10.596197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1980bb-fefe-4b64-ab0b-81f33b494790 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:10.596499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1980bb-fefe-4b64-ab0b-81f33b494790 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:10.597246 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1980bb-fefe-4b64-ab0b-81f33b494790 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:10.597594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1980bb-fefe-4b64-ab0b-81f33b494790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:10.599001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1980bb-fefe-4b64-ab0b-81f33b494790 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:10.609335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 627/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Jul 21 17:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:10.615143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a717dd-47aa-4233-a405-b74f5e00902d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:10.724793 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:10.743444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a717dd-47aa-4233-a405-b74f5e00902d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:10.781441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092bb847-cfc7-4f37-9d6a-4ab8264cd02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:10.782180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bb847-cfc7-4f37-9d6a-4ab8264cd02a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:10.782461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bb847-cfc7-4f37-9d6a-4ab8264cd02a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:10.782733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bb847-cfc7-4f37-9d6a-4ab8264cd02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:10.793833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092bb847-cfc7-4f37-9d6a-4ab8264cd02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:10.795262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092bb847-cfc7-4f37-9d6a-4ab8264cd02a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-092bb847-cfc7-4f37-9d6a-4ab8264cd02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:10.815138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:10.816152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:10.817124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:10.817124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:10.817124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:10.817124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:10.817124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 610/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:24:10.822897 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1980bb-fefe-4b64-ab0b-81f33b494790 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:10.939357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1980bb-fefe-4b64-ab0b-81f33b494790 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:10.950453 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:10.980742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00964318-5fdf-4958-9b42-7e96f48201d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:10.981563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00964318-5fdf-4958-9b42-7e96f48201d8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:10.981938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00964318-5fdf-4958-9b42-7e96f48201d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:10.984056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00964318-5fdf-4958-9b42-7e96f48201d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:10.984056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00964318-5fdf-4958-9b42-7e96f48201d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:10.984056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00964318-5fdf-4958-9b42-7e96f48201d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:10.985510 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00964318-5fdf-4958-9b42-7e96f48201d8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:10.997505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 628/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:11.003666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092bb847-cfc7-4f37-9d6a-4ab8264cd02a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:11.113891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092bb847-cfc7-4f37-9d6a-4ab8264cd02a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:11.150898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913d038-1d34-494e-bf3b-6502dbb8dfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:11.151690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913d038-1d34-494e-bf3b-6502dbb8dfdc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:11.151972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913d038-1d34-494e-bf3b-6502dbb8dfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:11.152249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913d038-1d34-494e-bf3b-6502dbb8dfdc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:11.156114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:11.163331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913d038-1d34-494e-bf3b-6502dbb8dfdc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:11.164849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913d038-1d34-494e-bf3b-6502dbb8dfdc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-3913d038-1d34-494e-bf3b-6502dbb8dfdc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:24:11.180529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:11.181889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:11.183037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 611/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:11.188067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00964318-5fdf-4958-9b42-7e96f48201d8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:11.304190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00964318-5fdf-4958-9b42-7e96f48201d8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:11.340221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e90927-3f53-4dc6-9867-5a92aefe84be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:11.340925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e90927-3f53-4dc6-9867-5a92aefe84be None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:11.341204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e90927-3f53-4dc6-9867-5a92aefe84be None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:11.341474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e90927-3f53-4dc6-9867-5a92aefe84be None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:11.342435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e90927-3f53-4dc6-9867-5a92aefe84be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:11.343298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e90927-3f53-4dc6-9867-5a92aefe84be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:11.344832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e90927-3f53-4dc6-9867-5a92aefe84be None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:11.365331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 629/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jul 21 17:24:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:24:11.371287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913d038-1d34-494e-bf3b-6502dbb8dfdc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:11.491568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913d038-1d34-494e-bf3b-6502dbb8dfdc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:11.532269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68364967-0697-4cc2-bb5e-13a567369b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:11.533136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68364967-0697-4cc2-bb5e-13a567369b25 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:11.533544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68364967-0697-4cc2-bb5e-13a567369b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:11.533990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68364967-0697-4cc2-bb5e-13a567369b25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:11.606654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:11.615539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68364967-0697-4cc2-bb5e-13a567369b25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:11.616998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68364967-0697-4cc2-bb5e-13a567369b25 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:11.730998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 612/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:24:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:24:11.738464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e90927-3f53-4dc6-9867-5a92aefe84be None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:11.771196 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:11.850000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e90927-3f53-4dc6-9867-5a92aefe84be None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:11.889613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdefa6e-6167-438a-b8bf-c97f96639385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:11.890959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdefa6e-6167-438a-b8bf-c97f96639385 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:11.891279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdefa6e-6167-438a-b8bf-c97f96639385 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:11.891547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdefa6e-6167-438a-b8bf-c97f96639385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:11.902561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdefa6e-6167-438a-b8bf-c97f96639385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:11.904008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdefa6e-6167-438a-b8bf-c97f96639385 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ccdefa6e-6167-438a-b8bf-c97f96639385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:11.922888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:11.925152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:11.927184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:11.927184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:11.927184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:11.927184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:11.929300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 630/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:11.930343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68364967-0697-4cc2-bb5e-13a567369b25 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:12.031274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:12.037857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68364967-0697-4cc2-bb5e-13a567369b25 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:12.074602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7af559-31b2-4142-af29-5dc29bec923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:12.075246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7af559-31b2-4142-af29-5dc29bec923c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:12.075535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7af559-31b2-4142-af29-5dc29bec923c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:12.075870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7af559-31b2-4142-af29-5dc29bec923c None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:12.076670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7af559-31b2-4142-af29-5dc29bec923c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:12.077527 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7af559-31b2-4142-af29-5dc29bec923c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:12.078880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7af559-31b2-4142-af29-5dc29bec923c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:12.092028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 613/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:24:12.098135 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdefa6e-6167-438a-b8bf-c97f96639385 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:12.183024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:12.207519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdefa6e-6167-438a-b8bf-c97f96639385 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:12.246791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b49f9-a22e-45b3-8e82-27a52cf3a460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:12.247673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b49f9-a22e-45b3-8e82-27a52cf3a460 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:12.247953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b49f9-a22e-45b3-8e82-27a52cf3a460 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:12.248240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b49f9-a22e-45b3-8e82-27a52cf3a460 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:12.257363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b49f9-a22e-45b3-8e82-27a52cf3a460 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:12.259216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b49f9-a22e-45b3-8e82-27a52cf3a460 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-e83b49f9-a22e-45b3-8e82-27a52cf3a460 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:24:12.278211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:12.279329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:12.280456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:24:12.280456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:24:12.280456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:24:12.280456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:24:12.280456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:24:12.280456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:24:12.280456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:12.280456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:12.280456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 631/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:12.285877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7af559-31b2-4142-af29-5dc29bec923c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:12.396323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7af559-31b2-4142-af29-5dc29bec923c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:12.432481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e616f3-8bd4-462f-a75f-6f59df58ff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:12.433263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e616f3-8bd4-462f-a75f-6f59df58ff5a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:12.433476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e616f3-8bd4-462f-a75f-6f59df58ff5a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:12.433781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e616f3-8bd4-462f-a75f-6f59df58ff5a None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:12.434583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e616f3-8bd4-462f-a75f-6f59df58ff5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:12.434899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e616f3-8bd4-462f-a75f-6f59df58ff5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:12.437181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e616f3-8bd4-462f-a75f-6f59df58ff5a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:12.453270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 614/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:24:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:12.459773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b49f9-a22e-45b3-8e82-27a52cf3a460 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:12.572312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b49f9-a22e-45b3-8e82-27a52cf3a460 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:12.609790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8d7f6-2ca9-4968-acbf-0acc1e0ae9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:12.610663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8d7f6-2ca9-4968-acbf-0acc1e0ae9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:12.611010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8d7f6-2ca9-4968-acbf-0acc1e0ae9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:12.611314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8d7f6-2ca9-4968-acbf-0acc1e0ae9e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:12.612155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8d7f6-2ca9-4968-acbf-0acc1e0ae9e0 None admin] RBAC: Adding query filter params (role.id=b9347810c7484efd95599fb9c51ab2df, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=901cf70064c34b63a0eda979a5ce77e4) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:12.612503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8d7f6-2ca9-4968-acbf-0acc1e0ae9e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:12.614140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8d7f6-2ca9-4968-acbf-0acc1e0ae9e0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:12.621612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 632/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:24:12 2022] GET /identity/v3/role_assignments?user.id=901cf70064c34b63a0eda979a5ce77e4&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=b9347810c7484efd95599fb9c51ab2df => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:15.049392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a358404-aa55-4c7b-a29c-0305aa209530 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:15.050106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a358404-aa55-4c7b-a29c-0305aa209530 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:15.050106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a358404-aa55-4c7b-a29c-0305aa209530 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:15.051397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 615/1247] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:24:15.058164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abe5f6-1cc7-4bb3-9c66-2e8f560fb919 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:15.058567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abe5f6-1cc7-4bb3-9c66-2e8f560fb919 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:15.058719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abe5f6-1cc7-4bb3-9c66-2e8f560fb919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:15.065362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:15.079990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-39abe5f6-1cc7-4bb3-9c66-2e8f560fb919 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:24:15.081233 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-39abe5f6-1cc7-4bb3-9c66-2e8f560fb919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:24:15.161295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 633/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:24:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:24:15.186831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a358404-aa55-4c7b-a29c-0305aa209530 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:15.208425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:15.313825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a358404-aa55-4c7b-a29c-0305aa209530 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:15.359806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fa85e-0b2f-4da9-a6c0-9acd40986e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:15.360635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fa85e-0b2f-4da9-a6c0-9acd40986e47 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:15.360916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fa85e-0b2f-4da9-a6c0-9acd40986e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:15.361189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fa85e-0b2f-4da9-a6c0-9acd40986e47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:15.373096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fa85e-0b2f-4da9-a6c0-9acd40986e47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-203fa85e-0b2f-4da9-a6c0-9acd40986e47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:24:15.381685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:15.382871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:15.384086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 616/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:24:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:24:15.389211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abe5f6-1cc7-4bb3-9c66-2e8f560fb919 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:15.507336 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abe5f6-1cc7-4bb3-9c66-2e8f560fb919 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:15.539374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:15.546634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db190935-ed30-4fd6-b77f-c6d6ee802f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:15.547315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190935-ed30-4fd6-b77f-c6d6ee802f20 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:15.547597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190935-ed30-4fd6-b77f-c6d6ee802f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:15.547872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190935-ed30-4fd6-b77f-c6d6ee802f20 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:15.548663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db190935-ed30-4fd6-b77f-c6d6ee802f20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:15.548971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db190935-ed30-4fd6-b77f-c6d6ee802f20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:15.550357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db190935-ed30-4fd6-b77f-c6d6ee802f20 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:15.559245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 634/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:24:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:15.566875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fa85e-0b2f-4da9-a6c0-9acd40986e47 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:15.668243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:15.703590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fa85e-0b2f-4da9-a6c0-9acd40986e47 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:15.747379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad9482-c9d3-41cf-a0d7-7623f4caa972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:15.748217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad9482-c9d3-41cf-a0d7-7623f4caa972 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:15.748566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad9482-c9d3-41cf-a0d7-7623f4caa972 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:15.748843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad9482-c9d3-41cf-a0d7-7623f4caa972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:15.761856 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad9482-c9d3-41cf-a0d7-7623f4caa972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:15.763529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad9482-c9d3-41cf-a0d7-7623f4caa972 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-58ad9482-c9d3-41cf-a0d7-7623f4caa972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:15.783176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:15.784282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:15.785228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:15.785228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:15.785228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:15.785228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:15.785228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 617/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:15.791173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db190935-ed30-4fd6-b77f-c6d6ee802f20 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:15.898462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db190935-ed30-4fd6-b77f-c6d6ee802f20 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:15.935106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba21966-3120-45f5-8c93-553869aafed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:15.935748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba21966-3120-45f5-8c93-553869aafed1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:15.936082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba21966-3120-45f5-8c93-553869aafed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:15.936946 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba21966-3120-45f5-8c93-553869aafed1 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:15.937695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba21966-3120-45f5-8c93-553869aafed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:15.938010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba21966-3120-45f5-8c93-553869aafed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:15.939396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba21966-3120-45f5-8c93-553869aafed1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:15.954339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 635/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:15.960056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad9482-c9d3-41cf-a0d7-7623f4caa972 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:16.085910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad9482-c9d3-41cf-a0d7-7623f4caa972 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:16.119829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:16.130217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d707e-9273-4e57-9843-21a7cd6f23b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:16.131304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d707e-9273-4e57-9843-21a7cd6f23b9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:16.131625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d707e-9273-4e57-9843-21a7cd6f23b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:16.131940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d707e-9273-4e57-9843-21a7cd6f23b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:16.141296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d707e-9273-4e57-9843-21a7cd6f23b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:16.143050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d707e-9273-4e57-9843-21a7cd6f23b9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-923d707e-9273-4e57-9843-21a7cd6f23b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:24:16.161289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:16.162282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:16.163440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 618/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:16.169045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba21966-3120-45f5-8c93-553869aafed1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:16.236822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:16.291189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba21966-3120-45f5-8c93-553869aafed1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:16.327569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0e168-e481-4a3c-8ca5-078dcf637b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:16.328289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0e168-e481-4a3c-8ca5-078dcf637b20 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:16.328639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0e168-e481-4a3c-8ca5-078dcf637b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:16.328938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0e168-e481-4a3c-8ca5-078dcf637b20 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:16.329784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0e168-e481-4a3c-8ca5-078dcf637b20 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:16.330098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0e168-e481-4a3c-8ca5-078dcf637b20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:16.331535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0e168-e481-4a3c-8ca5-078dcf637b20 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:16.349408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 636/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jul 21 17:24:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:16.355923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d707e-9273-4e57-9843-21a7cd6f23b9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:16.470587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d707e-9273-4e57-9843-21a7cd6f23b9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:16.510163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a091d-f49e-4630-8aec-23167368ab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:16.510932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a091d-f49e-4630-8aec-23167368ab59 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:16.511280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a091d-f49e-4630-8aec-23167368ab59 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:16.511558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a091d-f49e-4630-8aec-23167368ab59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:16.573948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:16.589035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a091d-f49e-4630-8aec-23167368ab59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:16.590696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a091d-f49e-4630-8aec-23167368ab59 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:16.716229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 619/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:24:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:24:16.722330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0e168-e481-4a3c-8ca5-078dcf637b20 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:16.840181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0e168-e481-4a3c-8ca5-078dcf637b20 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:16.855820 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:16.883745 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c15328-9480-49b9-88ed-009e31eaaa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:16.884585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c15328-9480-49b9-88ed-009e31eaaa64 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:16.884871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c15328-9480-49b9-88ed-009e31eaaa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:16.885193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c15328-9480-49b9-88ed-009e31eaaa64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:16.896695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c15328-9480-49b9-88ed-009e31eaaa64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:16.898192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c15328-9480-49b9-88ed-009e31eaaa64 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-86c15328-9480-49b9-88ed-009e31eaaa64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:16.915630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:16.916915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:16.917850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:16.917850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:16.917850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:16.917850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:16.917850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 637/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:16.923442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a091d-f49e-4630-8aec-23167368ab59 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:17.022681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:17.036224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a091d-f49e-4630-8aec-23167368ab59 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:17.076396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf05fb-b061-41e9-8349-22da0c3c965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:17.077333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf05fb-b061-41e9-8349-22da0c3c965f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:17.077713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf05fb-b061-41e9-8349-22da0c3c965f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:17.078096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf05fb-b061-41e9-8349-22da0c3c965f None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:17.079060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf05fb-b061-41e9-8349-22da0c3c965f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:17.079433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf05fb-b061-41e9-8349-22da0c3c965f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:17.081127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf05fb-b061-41e9-8349-22da0c3c965f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:17.101079 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 620/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:17.102350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c15328-9480-49b9-88ed-009e31eaaa64 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:17.213307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c15328-9480-49b9-88ed-009e31eaaa64 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:17.253596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77354f4b-37d5-482d-b27f-8301bccea67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:17.254408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77354f4b-37d5-482d-b27f-8301bccea67e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:17.254693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77354f4b-37d5-482d-b27f-8301bccea67e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:17.254971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77354f4b-37d5-482d-b27f-8301bccea67e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:17.262482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77354f4b-37d5-482d-b27f-8301bccea67e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:17.263962 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77354f4b-37d5-482d-b27f-8301bccea67e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-77354f4b-37d5-482d-b27f-8301bccea67e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:24:17.279118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:17.280850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:17.281803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:24:17.281803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:24:17.281803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:24:17.281803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:24:17.281803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:24:17.281803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:24:17.281803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:17.281803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:17.281803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 638/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:24:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 17:24:17.287229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf05fb-b061-41e9-8349-22da0c3c965f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:17.394534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf05fb-b061-41e9-8349-22da0c3c965f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:17.431360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b95dc-ebe8-4602-9e9b-8fa38a75e1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:17.432036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b95dc-ebe8-4602-9e9b-8fa38a75e1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:17.432340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b95dc-ebe8-4602-9e9b-8fa38a75e1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:17.432618 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b95dc-ebe8-4602-9e9b-8fa38a75e1f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:17.433418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b95dc-ebe8-4602-9e9b-8fa38a75e1f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:17.433743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b95dc-ebe8-4602-9e9b-8fa38a75e1f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:17.435172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b95dc-ebe8-4602-9e9b-8fa38a75e1f5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:17.453725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 621/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:17.454985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77354f4b-37d5-482d-b27f-8301bccea67e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:17.484510 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:17.564918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77354f4b-37d5-482d-b27f-8301bccea67e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:17.601438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35c9f0-239a-408a-9b9d-afe4cd40ffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:17.602100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35c9f0-239a-408a-9b9d-afe4cd40ffe2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:17.602380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35c9f0-239a-408a-9b9d-afe4cd40ffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:17.602655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35c9f0-239a-408a-9b9d-afe4cd40ffe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:17.603480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35c9f0-239a-408a-9b9d-afe4cd40ffe2 None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=901cf70064c34b63a0eda979a5ce77e4) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:17.603796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35c9f0-239a-408a-9b9d-afe4cd40ffe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:17.605247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35c9f0-239a-408a-9b9d-afe4cd40ffe2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:17.611541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 639/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:24:17 2022] GET /identity/v3/role_assignments?user.id=901cf70064c34b63a0eda979a5ce77e4&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:19.682071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d4e89-d4ee-4ce3-893c-56868a88a569 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:19.682692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d4e89-d4ee-4ce3-893c-56868a88a569 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:19.682692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d4e89-d4ee-4ce3-893c-56868a88a569 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:19.684216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 622/1261] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:24:19.691197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469d19a-c191-4af3-b191-9aeb87393e66 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:19.691600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469d19a-c191-4af3-b191-9aeb87393e66 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:19.691807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469d19a-c191-4af3-b191-9aeb87393e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:19.709582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7469d19a-c191-4af3-b191-9aeb87393e66 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:24:19.710755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7469d19a-c191-4af3-b191-9aeb87393e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:24:19.776130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 640/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:24:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:24:19.800070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d4e89-d4ee-4ce3-893c-56868a88a569 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:19.810732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:19.900413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d4e89-d4ee-4ce3-893c-56868a88a569 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:19.934621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082108b-5363-42a8-9c20-546ae2748cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:19.935417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082108b-5363-42a8-9c20-546ae2748cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:19.935698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082108b-5363-42a8-9c20-546ae2748cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:19.935981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082108b-5363-42a8-9c20-546ae2748cc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:19.947491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082108b-5363-42a8-9c20-546ae2748cc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-2082108b-5363-42a8-9c20-546ae2748cc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:24:19.955324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:19.956251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:19.957131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 623/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:24:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:19.962563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469d19a-c191-4af3-b191-9aeb87393e66 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:19.980541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:20.060031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469d19a-c191-4af3-b191-9aeb87393e66 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:20.094284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380e0a8-b684-401c-bc3f-d0e295ab2565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:20.094934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380e0a8-b684-401c-bc3f-d0e295ab2565 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:20.095217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380e0a8-b684-401c-bc3f-d0e295ab2565 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:20.095492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380e0a8-b684-401c-bc3f-d0e295ab2565 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:20.096230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380e0a8-b684-401c-bc3f-d0e295ab2565 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:20.096566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380e0a8-b684-401c-bc3f-d0e295ab2565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:20.097954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380e0a8-b684-401c-bc3f-d0e295ab2565 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:20.105744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 641/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:24:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:20.112075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082108b-5363-42a8-9c20-546ae2748cc5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:20.179250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:20.216445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082108b-5363-42a8-9c20-546ae2748cc5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:20.250480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce4c34-9a3c-465d-9fea-0ecfda8e7d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:20.251271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce4c34-9a3c-465d-9fea-0ecfda8e7d02 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:20.251556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce4c34-9a3c-465d-9fea-0ecfda8e7d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:20.251835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce4c34-9a3c-465d-9fea-0ecfda8e7d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:20.262005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce4c34-9a3c-465d-9fea-0ecfda8e7d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:20.263442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce4c34-9a3c-465d-9fea-0ecfda8e7d02 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-b1ce4c34-9a3c-465d-9fea-0ecfda8e7d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:20.280939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:20.281966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:20.283077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:20.283077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:20.283077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:20.283077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:20.283077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 624/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:20.287920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380e0a8-b684-401c-bc3f-d0e295ab2565 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:20.366484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:20.385547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380e0a8-b684-401c-bc3f-d0e295ab2565 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:20.418585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca418cf-def5-489f-9010-6820359eaf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:20.419225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca418cf-def5-489f-9010-6820359eaf1d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:20.419506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca418cf-def5-489f-9010-6820359eaf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:20.419779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca418cf-def5-489f-9010-6820359eaf1d None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:20.421148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca418cf-def5-489f-9010-6820359eaf1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:20.421449 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca418cf-def5-489f-9010-6820359eaf1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:20.422803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca418cf-def5-489f-9010-6820359eaf1d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:20.434098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 642/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:20.439707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce4c34-9a3c-465d-9fea-0ecfda8e7d02 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:20.540383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce4c34-9a3c-465d-9fea-0ecfda8e7d02 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:20.559465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:20.574068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a544f7-9741-4d99-980c-5787a8f96204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:20.574853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a544f7-9741-4d99-980c-5787a8f96204 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:20.575135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a544f7-9741-4d99-980c-5787a8f96204 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:20.575409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a544f7-9741-4d99-980c-5787a8f96204 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:20.584440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a544f7-9741-4d99-980c-5787a8f96204 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:20.585873 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a544f7-9741-4d99-980c-5787a8f96204 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e2a544f7-9741-4d99-980c-5787a8f96204 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:24:20.600735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:20.601847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:20.603001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 625/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:20.607667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca418cf-def5-489f-9010-6820359eaf1d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:20.718855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca418cf-def5-489f-9010-6820359eaf1d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:20.750796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa80c7c-e595-4c99-9d93-3000e3838878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:20.751449 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa80c7c-e595-4c99-9d93-3000e3838878 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:20.751727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa80c7c-e595-4c99-9d93-3000e3838878 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:20.751998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa80c7c-e595-4c99-9d93-3000e3838878 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:20.752852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa80c7c-e595-4c99-9d93-3000e3838878 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:20.753159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa80c7c-e595-4c99-9d93-3000e3838878 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:20.754600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa80c7c-e595-4c99-9d93-3000e3838878 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:20.762806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:20.772205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 643/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jul 21 17:24:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:20.777874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a544f7-9741-4d99-980c-5787a8f96204 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:20.875731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a544f7-9741-4d99-980c-5787a8f96204 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:20.907899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e2210-1714-4e0b-9267-33f0dd0dc921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:20.908691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e2210-1714-4e0b-9267-33f0dd0dc921 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:20.908970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e2210-1714-4e0b-9267-33f0dd0dc921 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:20.909239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e2210-1714-4e0b-9267-33f0dd0dc921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:20.919528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e2210-1714-4e0b-9267-33f0dd0dc921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:20.921122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e2210-1714-4e0b-9267-33f0dd0dc921 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8e6e2210-1714-4e0b-9267-33f0dd0dc921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:20.938434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:20.939472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:20.940586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:20.940586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:20.940586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:20.940586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:20.940586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 626/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:20.945476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa80c7c-e595-4c99-9d93-3000e3838878 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:21.041011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa80c7c-e595-4c99-9d93-3000e3838878 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:21.074045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f703e8-df73-4289-b7dd-7a4ba1bd8e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:21.074706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f703e8-df73-4289-b7dd-7a4ba1bd8e7a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:21.074969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f703e8-df73-4289-b7dd-7a4ba1bd8e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:21.075240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f703e8-df73-4289-b7dd-7a4ba1bd8e7a None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:21.075945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f703e8-df73-4289-b7dd-7a4ba1bd8e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:21.076266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f703e8-df73-4289-b7dd-7a4ba1bd8e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:21.078248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f703e8-df73-4289-b7dd-7a4ba1bd8e7a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:21.088992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 644/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:21.094637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e2210-1714-4e0b-9267-33f0dd0dc921 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:21.125367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:21.190955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e2210-1714-4e0b-9267-33f0dd0dc921 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:21.223627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2372c1c-daae-4af7-b806-4b8f6a79b418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:21.224434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2372c1c-daae-4af7-b806-4b8f6a79b418 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:21.224714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2372c1c-daae-4af7-b806-4b8f6a79b418 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:21.224983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2372c1c-daae-4af7-b806-4b8f6a79b418 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:21.231362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2372c1c-daae-4af7-b806-4b8f6a79b418 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:21.233467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2372c1c-daae-4af7-b806-4b8f6a79b418 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f2372c1c-daae-4af7-b806-4b8f6a79b418 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:24:21.247231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:21.248992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:21.250289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:24:21.250289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:24:21.250289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:24:21.250289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:24:21.250289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:24:21.250289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:24:21.250289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:21.250289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:21.250289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 627/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:21.254783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f703e8-df73-4289-b7dd-7a4ba1bd8e7a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:21.311952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:21.352054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f703e8-df73-4289-b7dd-7a4ba1bd8e7a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:21.385149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86668585-7115-4e7d-81ae-214d977d386e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:21.385790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668585-7115-4e7d-81ae-214d977d386e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:21.386070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668585-7115-4e7d-81ae-214d977d386e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:21.386344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668585-7115-4e7d-81ae-214d977d386e None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:21.387119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86668585-7115-4e7d-81ae-214d977d386e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:21.387432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86668585-7115-4e7d-81ae-214d977d386e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:21.388915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86668585-7115-4e7d-81ae-214d977d386e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:21.400606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 645/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:21.406682 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2372c1c-daae-4af7-b806-4b8f6a79b418 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:21.497199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:21.506759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2372c1c-daae-4af7-b806-4b8f6a79b418 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:21.539021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee5268-de55-4a65-9f6a-10490e03d9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:21.539887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee5268-de55-4a65-9f6a-10490e03d9de None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:21.540160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee5268-de55-4a65-9f6a-10490e03d9de None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:21.540458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee5268-de55-4a65-9f6a-10490e03d9de None admin] PATH_INFO: `/v3/projects/66d87a2609e44063a774cbfc545412dc/users/901cf70064c34b63a0eda979a5ce77e4/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:21.542357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee5268-de55-4a65-9f6a-10490e03d9de None admin] RBAC: Authorizing `identity:create_grant(project_id=66d87a2609e44063a774cbfc545412dc, user_id=901cf70064c34b63a0eda979a5ce77e4, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:21.543765 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee5268-de55-4a65-9f6a-10490e03d9de None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:21.556073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 628/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Jul 21 17:24:21 2022] PUT /identity/v3/projects/66d87a2609e44063a774cbfc545412dc/users/901cf70064c34b63a0eda979a5ce77e4/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:24:23.542294 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc0dd8-6f46-4e23-9dfb-e77830faff88 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:23.542846 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc0dd8-6f46-4e23-9dfb-e77830faff88 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:23.542846 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc0dd8-6f46-4e23-9dfb-e77830faff88 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:23.544191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 646/1274] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:24:23.549921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74dab3-546c-4ce0-95f2-328c1b220ae3 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:23.550344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74dab3-546c-4ce0-95f2-328c1b220ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:23.550445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74dab3-546c-4ce0-95f2-328c1b220ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:23.567233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5c74dab3-546c-4ce0-95f2-328c1b220ae3 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:24:23.568439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5c74dab3-546c-4ce0-95f2-328c1b220ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:24:23.635258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 629/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:24:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 17:24:23.662751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc0dd8-6f46-4e23-9dfb-e77830faff88 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:23.760465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc0dd8-6f46-4e23-9dfb-e77830faff88 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:23.776851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:23.793388 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff15e5-800c-4392-bc0f-8c9f9cd58de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:23.794107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff15e5-800c-4392-bc0f-8c9f9cd58de8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:23.794392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff15e5-800c-4392-bc0f-8c9f9cd58de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:23.794667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff15e5-800c-4392-bc0f-8c9f9cd58de8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:23.805202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff15e5-800c-4392-bc0f-8c9f9cd58de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-37ff15e5-800c-4392-bc0f-8c9f9cd58de8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 17:24:23.813506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:23.814623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:23.815650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 647/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Jul 21 17:24:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:23.820866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74dab3-546c-4ce0-95f2-328c1b220ae3 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:23.896993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:23.915450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74dab3-546c-4ce0-95f2-328c1b220ae3 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:23.949822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afed754-15a1-4b39-b30d-9d2001e2d427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:23.950494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afed754-15a1-4b39-b30d-9d2001e2d427 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:23.950775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afed754-15a1-4b39-b30d-9d2001e2d427 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:23.951055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afed754-15a1-4b39-b30d-9d2001e2d427 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:23.951838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afed754-15a1-4b39-b30d-9d2001e2d427 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:23.952143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afed754-15a1-4b39-b30d-9d2001e2d427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:23.953539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afed754-15a1-4b39-b30d-9d2001e2d427 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:23.961414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 630/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Jul 21 17:24:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:23.967710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff15e5-800c-4392-bc0f-8c9f9cd58de8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:24.064345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff15e5-800c-4392-bc0f-8c9f9cd58de8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:24.097234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f629183-c53e-456c-8639-14886fad38f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:24.098004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f629183-c53e-456c-8639-14886fad38f0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:24.098287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f629183-c53e-456c-8639-14886fad38f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:24.098563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f629183-c53e-456c-8639-14886fad38f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:24.109525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f629183-c53e-456c-8639-14886fad38f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:24.110962 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f629183-c53e-456c-8639-14886fad38f0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-7f629183-c53e-456c-8639-14886fad38f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:24.127623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:24.128658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:24.129621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:24.129621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:24.129621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:24.129621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:24.129621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 648/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:24.134349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afed754-15a1-4b39-b30d-9d2001e2d427 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:24.230411 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afed754-15a1-4b39-b30d-9d2001e2d427 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:24.262332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ced23b-a114-4b58-be75-84207e470b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:24.262979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ced23b-a114-4b58-be75-84207e470b07 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:24.263264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ced23b-a114-4b58-be75-84207e470b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:24.264260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ced23b-a114-4b58-be75-84207e470b07 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:24.265179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ced23b-a114-4b58-be75-84207e470b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:24.265489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ced23b-a114-4b58-be75-84207e470b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:24.266834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ced23b-a114-4b58-be75-84207e470b07 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:24.278059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 631/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:24.283606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f629183-c53e-456c-8639-14886fad38f0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:24.314302 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:24.382740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f629183-c53e-456c-8639-14886fad38f0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:24.414812 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2da072-1e79-46f4-a71a-f3494768c4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:24.415594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2da072-1e79-46f4-a71a-f3494768c4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:24.415876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2da072-1e79-46f4-a71a-f3494768c4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:24.416151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2da072-1e79-46f4-a71a-f3494768c4f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:24.424099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2da072-1e79-46f4-a71a-f3494768c4f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:24.425557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2da072-1e79-46f4-a71a-f3494768c4f3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-4f2da072-1e79-46f4-a71a-f3494768c4f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 17:24:24.439903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:24.441095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:24.442295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 649/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:24.447115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ced23b-a114-4b58-be75-84207e470b07 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:24.450875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:24.543900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ced23b-a114-4b58-be75-84207e470b07 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:24.577287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2c37-06bf-40d6-bb65-71566d204a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:24.577949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2c37-06bf-40d6-bb65-71566d204a2b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:24.578235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2c37-06bf-40d6-bb65-71566d204a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:24.578517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2c37-06bf-40d6-bb65-71566d204a2b None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:24.579349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2c37-06bf-40d6-bb65-71566d204a2b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:24.579660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2c37-06bf-40d6-bb65-71566d204a2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:24.581107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2c37-06bf-40d6-bb65-71566d204a2b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:24.597653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 632/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Jul 21 17:24:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:24.603245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2da072-1e79-46f4-a71a-f3494768c4f3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:24.696443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:24.705828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2da072-1e79-46f4-a71a-f3494768c4f3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:24.738722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3cef7-65c7-4f94-a376-697521378d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:24.739363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3cef7-65c7-4f94-a376-697521378d90 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:24.739646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3cef7-65c7-4f94-a376-697521378d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:24.739922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3cef7-65c7-4f94-a376-697521378d90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:24.804091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3cef7-65c7-4f94-a376-697521378d90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:24.805545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3cef7-65c7-4f94-a376-697521378d90 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:24.890058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:24.900170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 650/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Jul 21 17:24:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 17:24:24.906431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2c37-06bf-40d6-bb65-71566d204a2b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:24.967425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:25.004696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2c37-06bf-40d6-bb65-71566d204a2b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:25.037764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7262fa-d0c0-4a18-93b5-694b2b875f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:25.038540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7262fa-d0c0-4a18-93b5-694b2b875f92 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:25.038819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7262fa-d0c0-4a18-93b5-694b2b875f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:25.039089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7262fa-d0c0-4a18-93b5-694b2b875f92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:25.049148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7262fa-d0c0-4a18-93b5-694b2b875f92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:25.050572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7262fa-d0c0-4a18-93b5-694b2b875f92 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-fd7262fa-d0c0-4a18-93b5-694b2b875f92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 17:24:25.067504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:25.068807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:25.069726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 17:24:25.069726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 17:24:25.069726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 17:24:25.069726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:25.069726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 633/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:25.074653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3cef7-65c7-4f94-a376-697521378d90 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:25.171555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3cef7-65c7-4f94-a376-697521378d90 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:25.204361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35f54-a716-4e01-a64b-41eb285671b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:25.205006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35f54-a716-4e01-a64b-41eb285671b5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:25.205291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35f54-a716-4e01-a64b-41eb285671b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:25.205569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35f54-a716-4e01-a64b-41eb285671b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:25.206280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb35f54-a716-4e01-a64b-41eb285671b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:25.206588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb35f54-a716-4e01-a64b-41eb285671b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:25.207955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb35f54-a716-4e01-a64b-41eb285671b5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:25.219575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 651/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Jul 21 17:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:25.225104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7262fa-d0c0-4a18-93b5-694b2b875f92 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:25.320752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7262fa-d0c0-4a18-93b5-694b2b875f92 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:25.334754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:25.354710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17576c-219b-4231-ade8-a5469785c5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:25.355484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17576c-219b-4231-ade8-a5469785c5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:25.355761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17576c-219b-4231-ade8-a5469785c5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:25.356031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17576c-219b-4231-ade8-a5469785c5c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:25.362529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17576c-219b-4231-ade8-a5469785c5c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:25.363950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17576c-219b-4231-ade8-a5469785c5c3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-4d17576c-219b-4231-ade8-a5469785c5c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 17:24:25.377784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:25.378826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:25.379930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 17:24:25.379930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 17:24:25.379930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 17:24:25.379930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 17:24:25.379930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 17:24:25.379930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 17:24:25.379930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 17:24:25.379930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 17:24:25.379930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 634/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Jul 21 17:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:25.384853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35f54-a716-4e01-a64b-41eb285671b5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:25.432558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:25.480685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35f54-a716-4e01-a64b-41eb285671b5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:25.523632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503a3c8-d694-42b3-b0ff-b0e091c710ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:25.524285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503a3c8-d694-42b3-b0ff-b0e091c710ff None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:25.524596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503a3c8-d694-42b3-b0ff-b0e091c710ff None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:25.524874 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503a3c8-d694-42b3-b0ff-b0e091c710ff None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:25.525656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503a3c8-d694-42b3-b0ff-b0e091c710ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:25.525969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503a3c8-d694-42b3-b0ff-b0e091c710ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:25.527405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503a3c8-d694-42b3-b0ff-b0e091c710ff None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:25.539222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 652/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Jul 21 17:24:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:25.545514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17576c-219b-4231-ade8-a5469785c5c3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:25.649967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17576c-219b-4231-ade8-a5469785c5c3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:25.682390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f73572-4e9d-40e1-a3d2-17f851265e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:25.683035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f73572-4e9d-40e1-a3d2-17f851265e1f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:25.683314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f73572-4e9d-40e1-a3d2-17f851265e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:25.683587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f73572-4e9d-40e1-a3d2-17f851265e1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:25.684506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f73572-4e9d-40e1-a3d2-17f851265e1f None admin] RBAC: Adding query filter params (role.id=2ea917a7fe2a4acfaca8789f3c8f9857, scope.project.id=66d87a2609e44063a774cbfc545412dc, user.id=901cf70064c34b63a0eda979a5ce77e4) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:25.684815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f73572-4e9d-40e1-a3d2-17f851265e1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:25.686933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f73572-4e9d-40e1-a3d2-17f851265e1f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:25.692586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 635/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Jul 21 17:24:25 2022] GET /identity/v3/role_assignments?user.id=901cf70064c34b63a0eda979a5ce77e4&scope.project.id=66d87a2609e44063a774cbfc545412dc&role.id=2ea917a7fe2a4acfaca8789f3c8f9857 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:27.750220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c652ac6-6ddf-482c-b9a5-848c15db1e60 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:27.750857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c652ac6-6ddf-482c-b9a5-848c15db1e60 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:27.750857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c652ac6-6ddf-482c-b9a5-848c15db1e60 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:27.752127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 653/1288] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:24:27.757113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97800fc3-8efd-47c5-9f43-6a08f52ed4f4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:27.757500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97800fc3-8efd-47c5-9f43-6a08f52ed4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:27.757661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97800fc3-8efd-47c5-9f43-6a08f52ed4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:27.774406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-97800fc3-8efd-47c5-9f43-6a08f52ed4f4 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:24:27.775566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-97800fc3-8efd-47c5-9f43-6a08f52ed4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:24:27.803891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:27.842422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 636/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:24:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:24:27.864176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c652ac6-6ddf-482c-b9a5-848c15db1e60 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:27.963986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c652ac6-6ddf-482c-b9a5-848c15db1e60 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:27.974896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:27.998448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202757cd-02b3-4c7a-a16c-d2688aacdf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:27.999229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202757cd-02b3-4c7a-a16c-d2688aacdf27 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:27.999518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202757cd-02b3-4c7a-a16c-d2688aacdf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:27.999790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202757cd-02b3-4c7a-a16c-d2688aacdf27 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:28.000517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202757cd-02b3-4c7a-a16c-d2688aacdf27 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:28.001839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202757cd-02b3-4c7a-a16c-d2688aacdf27 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-202757cd-02b3-4c7a-a16c-d2688aacdf27 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:28.013417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:28.014683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:28.015899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 654/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jul 21 17:24:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:28.020906 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97800fc3-8efd-47c5-9f43-6a08f52ed4f4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:28.115181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97800fc3-8efd-47c5-9f43-6a08f52ed4f4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:28.147414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed8b56-0ba5-4db4-81b0-3818e719bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:28.148062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed8b56-0ba5-4db4-81b0-3818e719bc4a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:28.148436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed8b56-0ba5-4db4-81b0-3818e719bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:28.148715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed8b56-0ba5-4db4-81b0-3818e719bc4a None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:28.149410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed8b56-0ba5-4db4-81b0-3818e719bc4a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:28.149715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed8b56-0ba5-4db4-81b0-3818e719bc4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:28.151063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed8b56-0ba5-4db4-81b0-3818e719bc4a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:28.155867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 637/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Jul 21 17:24:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:24:28.161578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202757cd-02b3-4c7a-a16c-d2688aacdf27 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:28.258174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202757cd-02b3-4c7a-a16c-d2688aacdf27 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:28.289482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b20333-f9cd-4ca4-97cb-3b79a4a2134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:28.290123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b20333-f9cd-4ca4-97cb-3b79a4a2134f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:28.290403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b20333-f9cd-4ca4-97cb-3b79a4a2134f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:28.290676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b20333-f9cd-4ca4-97cb-3b79a4a2134f None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:28.291367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b20333-f9cd-4ca4-97cb-3b79a4a2134f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:28.291670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b20333-f9cd-4ca4-97cb-3b79a4a2134f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:28.293048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b20333-f9cd-4ca4-97cb-3b79a4a2134f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:28.298015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 655/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Jul 21 17:24:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:24:30.374230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1af251-bd6c-44d5-832e-c152dd7e3ad8 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:30.374764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1af251-bd6c-44d5-832e-c152dd7e3ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:30.374868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1af251-bd6c-44d5-832e-c152dd7e3ad8 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:30.376198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 638/1293] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:24:30.381298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18854f-b8a1-48cb-9ceb-65ecc721f281 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:30.381744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18854f-b8a1-48cb-9ceb-65ecc721f281 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:30.381842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18854f-b8a1-48cb-9ceb-65ecc721f281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:30.399437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-dd18854f-b8a1-48cb-9ceb-65ecc721f281 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:24:30.400647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dd18854f-b8a1-48cb-9ceb-65ecc721f281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:24:30.429410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:30.466123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 656/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:24:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:24:30.488653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1af251-bd6c-44d5-832e-c152dd7e3ad8 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:30.518683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:30.585819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1af251-bd6c-44d5-832e-c152dd7e3ad8 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:30.619360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bceffe-e5f2-408a-a606-4e9c5cd6a9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:30.620013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bceffe-e5f2-408a-a606-4e9c5cd6a9c7 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:30.620319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bceffe-e5f2-408a-a606-4e9c5cd6a9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:30.620596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bceffe-e5f2-408a-a606-4e9c5cd6a9c7 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:30.621308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bceffe-e5f2-408a-a606-4e9c5cd6a9c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:30.623673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bceffe-e5f2-408a-a606-4e9c5cd6a9c7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:30.637367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 639/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Jul 21 17:24:30 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:24:32.726284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7519f-6bc9-4b01-96de-de6b147f9f3e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:32.726839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7519f-6bc9-4b01-96de-de6b147f9f3e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:32.726839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7519f-6bc9-4b01-96de-de6b147f9f3e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:32.728216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 657/1296] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:24:32.733473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cd938-3258-44af-8b79-46488a78714d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:32.733901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cd938-3258-44af-8b79-46488a78714d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:32.734005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cd938-3258-44af-8b79-46488a78714d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:32.750267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d96cd938-3258-44af-8b79-46488a78714d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:24:32.751434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d96cd938-3258-44af-8b79-46488a78714d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:24:32.821064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 640/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:24:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:24:32.844193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7519f-6bc9-4b01-96de-de6b147f9f3e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:32.944138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7519f-6bc9-4b01-96de-de6b147f9f3e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:32.976696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53fa57-a5f1-4a8e-8958-68f19948a9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:32.977417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53fa57-a5f1-4a8e-8958-68f19948a9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:32.977695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53fa57-a5f1-4a8e-8958-68f19948a9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:32.977963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53fa57-a5f1-4a8e-8958-68f19948a9e4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:32.978607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53fa57-a5f1-4a8e-8958-68f19948a9e4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:32.979961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53fa57-a5f1-4a8e-8958-68f19948a9e4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-7a53fa57-a5f1-4a8e-8958-68f19948a9e4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:32.991460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:32.992676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:32.993795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 658/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jul 21 17:24:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:32.999182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cd938-3258-44af-8b79-46488a78714d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:33.017561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:33.095309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cd938-3258-44af-8b79-46488a78714d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:33.127731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f4cb7-3f89-4015-bfdb-1b1d8f3025a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:33.128417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f4cb7-3f89-4015-bfdb-1b1d8f3025a1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:33.128703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f4cb7-3f89-4015-bfdb-1b1d8f3025a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:33.128979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f4cb7-3f89-4015-bfdb-1b1d8f3025a1 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:33.129746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f4cb7-3f89-4015-bfdb-1b1d8f3025a1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:33.130049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f4cb7-3f89-4015-bfdb-1b1d8f3025a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:33.131418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f4cb7-3f89-4015-bfdb-1b1d8f3025a1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:33.136727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 641/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Jul 21 17:24:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:33.142742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53fa57-a5f1-4a8e-8958-68f19948a9e4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:33.174410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:33.240507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53fa57-a5f1-4a8e-8958-68f19948a9e4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:33.272249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05345fbe-b11b-4a64-84eb-9fd8a9a8e8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:33.272917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05345fbe-b11b-4a64-84eb-9fd8a9a8e8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:33.273197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05345fbe-b11b-4a64-84eb-9fd8a9a8e8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:33.273468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05345fbe-b11b-4a64-84eb-9fd8a9a8e8b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:33.274239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05345fbe-b11b-4a64-84eb-9fd8a9a8e8b4 None admin] RBAC: Adding query filter params (interface=public, service_id=a98f08823a01498c9055908eb898b7b4, region_id=RegionOne) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:33.274631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05345fbe-b11b-4a64-84eb-9fd8a9a8e8b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:33.275988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05345fbe-b11b-4a64-84eb-9fd8a9a8e8b4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:33.281477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 659/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Jul 21 17:24:33 2022] GET /identity/v3/endpoints?service_id=a98f08823a01498c9055908eb898b7b4&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:33.287027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f4cb7-3f89-4015-bfdb-1b1d8f3025a1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:33.366160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:33.383430 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f4cb7-3f89-4015-bfdb-1b1d8f3025a1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:33.416306 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c238c3-bb2d-4e2e-be41-a5b7c279296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:33.416978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c238c3-bb2d-4e2e-be41-a5b7c279296e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:33.417258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c238c3-bb2d-4e2e-be41-a5b7c279296e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:33.417529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c238c3-bb2d-4e2e-be41-a5b7c279296e None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:33.418201 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c238c3-bb2d-4e2e-be41-a5b7c279296e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:33.420268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c238c3-bb2d-4e2e-be41-a5b7c279296e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:33.426964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 642/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Jul 21 17:24:33 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:24:35.417440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344e09-0f25-44bc-a61e-aab738b84219 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:35.418001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344e09-0f25-44bc-a61e-aab738b84219 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:35.418001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344e09-0f25-44bc-a61e-aab738b84219 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:35.420072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 660/1302] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:24:35.425215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248135e-22a6-4580-8598-a7006f3f55d5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:35.425603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248135e-22a6-4580-8598-a7006f3f55d5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:35.425772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248135e-22a6-4580-8598-a7006f3f55d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:35.442732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4248135e-22a6-4580-8598-a7006f3f55d5 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:24:35.444178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4248135e-22a6-4580-8598-a7006f3f55d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:24:35.509470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 643/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:24:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 17:24:35.533297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3344e09-0f25-44bc-a61e-aab738b84219 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:35.632092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:35.639815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3344e09-0f25-44bc-a61e-aab738b84219 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:35.672508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080802d4-d860-4738-8cf9-5a53c8708e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:35.673235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080802d4-d860-4738-8cf9-5a53c8708e40 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:35.673524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080802d4-d860-4738-8cf9-5a53c8708e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:35.673805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080802d4-d860-4738-8cf9-5a53c8708e40 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:35.674452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080802d4-d860-4738-8cf9-5a53c8708e40 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:35.675817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080802d4-d860-4738-8cf9-5a53c8708e40 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-080802d4-d860-4738-8cf9-5a53c8708e40 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 17:24:35.687553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 17:24:35.688796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 21 17:24:35.690080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 661/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Jul 21 17:24:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 17:24:35.695032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248135e-22a6-4580-8598-a7006f3f55d5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:35.770185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:35.792730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248135e-22a6-4580-8598-a7006f3f55d5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:35.827285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1aa958-7089-4db4-bf70-f79cc809d2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:35.827946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1aa958-7089-4db4-bf70-f79cc809d2fe None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:35.828234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1aa958-7089-4db4-bf70-f79cc809d2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:35.828545 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1aa958-7089-4db4-bf70-f79cc809d2fe None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:35.829254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1aa958-7089-4db4-bf70-f79cc809d2fe None admin] RBAC: Adding query filter params (name=placement) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:24:35.829565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1aa958-7089-4db4-bf70-f79cc809d2fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:35.830926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1aa958-7089-4db4-bf70-f79cc809d2fe None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:35.835990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 644/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Jul 21 17:24:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:24:35.842681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080802d4-d860-4738-8cf9-5a53c8708e40 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:24:35.940459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080802d4-d860-4738-8cf9-5a53c8708e40 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:24:35.973632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e603ac-3ccc-4607-9ef4-90d98ae165cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:24:35.974308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e603ac-3ccc-4607-9ef4-90d98ae165cf None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:24:35.974592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e603ac-3ccc-4607-9ef4-90d98ae165cf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:24:35.974870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e603ac-3ccc-4607-9ef4-90d98ae165cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:24:35.975603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e603ac-3ccc-4607-9ef4-90d98ae165cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:24:35.977003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e603ac-3ccc-4607-9ef4-90d98ae165cf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:24:35.994595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 662/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Jul 21 17:24:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:24:45.832977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=30 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:24:45.990173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:39.242016 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87548a-40cc-4966-b1c7-82fe0e717f1d None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:39.242664 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87548a-40cc-4966-b1c7-82fe0e717f1d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:39.242664 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87548a-40cc-4966-b1c7-82fe0e717f1d None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:39.249446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647172f-c622-4332-8094-00f230543856 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:39.249909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647172f-c622-4332-8094-00f230543856 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:39.250005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647172f-c622-4332-8094-00f230543856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:39.253518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 645/1307] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:25:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:25:39.338941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e647172f-c622-4332-8094-00f230543856 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:25:39.340939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e647172f-c622-4332-8094-00f230543856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:39.469689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 663/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:25:39.526174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec971b-1af8-402a-8684-75a6b656293b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:39.526535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec971b-1af8-402a-8684-75a6b656293b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:39.526898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec971b-1af8-402a-8684-75a6b656293b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:39.529251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 646/1309] 10.222.0.57 () {54 vars in 965 bytes} [Thu Jul 21 17:25:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:25:39.545940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32070d1c-f938-4138-b912-59560308a0c1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:39.546146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32070d1c-f938-4138-b912-59560308a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:39.546432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32070d1c-f938-4138-b912-59560308a0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:39.670607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-32070d1c-f938-4138-b912-59560308a0c1 None None] MFA Rules not processed for user `b0edda384caf46819169b69b858afc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:25:39.671800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-32070d1c-f938-4138-b912-59560308a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:39.695093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:39.754091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 664/1310] 10.222.0.57 () {58 vars in 1062 bytes} [Thu Jul 21 17:25:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:25:39.763313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec971b-1af8-402a-8684-75a6b656293b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:25:39.765413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-22ec971b-1af8-402a-8684-75a6b656293b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:39.884398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec971b-1af8-402a-8684-75a6b656293b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:25:39.969851 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d4331-94da-42af-a3d0-2f894f628b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:25:39.970512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d4331-94da-42af-a3d0-2f894f628b7f service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:39.970794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d4331-94da-42af-a3d0-2f894f628b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:39.971068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d4331-94da-42af-a3d0-2f894f628b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:39.972776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-942d4331-94da-42af-a3d0-2f894f628b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:40.031052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:40.092175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d4331-94da-42af-a3d0-2f894f628b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:25:40.093778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d4331-94da-42af-a3d0-2f894f628b7f service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:25:40.114208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 647/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:25:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:25:42.468137 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b860a6-c3ae-40af-b503-d9c89fa0f291 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:42.468705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b860a6-c3ae-40af-b503-d9c89fa0f291 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:42.468705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b860a6-c3ae-40af-b503-d9c89fa0f291 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:42.470242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 665/1312] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:25:42.475349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa12b6-a8ad-459f-8146-fe11c53ffe58 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:42.475746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa12b6-a8ad-459f-8146-fe11c53ffe58 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:42.475950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa12b6-a8ad-459f-8146-fe11c53ffe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:42.558150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-10aa12b6-a8ad-459f-8146-fe11c53ffe58 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:25:42.559359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-10aa12b6-a8ad-459f-8146-fe11c53ffe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:42.660028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 648/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:25:42.698806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b860a6-c3ae-40af-b503-d9c89fa0f291 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:25:42.700659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-88b860a6-c3ae-40af-b503-d9c89fa0f291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:42.824853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b860a6-c3ae-40af-b503-d9c89fa0f291 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:25:42.909137 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:42.912069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c90be9-7e2c-46fc-9aed-b3ce7605d8e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:25:42.915829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c90be9-7e2c-46fc-9aed-b3ce7605d8e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:42.915829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c90be9-7e2c-46fc-9aed-b3ce7605d8e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:42.915829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c90be9-7e2c-46fc-9aed-b3ce7605d8e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:42.915829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f4c90be9-7e2c-46fc-9aed-b3ce7605d8e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:43.047562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c90be9-7e2c-46fc-9aed-b3ce7605d8e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:25:43.053880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c90be9-7e2c-46fc-9aed-b3ce7605d8e5 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:25:43.075465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 666/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:25:45.520315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8188cc5-74cc-4a83-bffc-0ff381371492 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:45.520874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8188cc5-74cc-4a83-bffc-0ff381371492 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:45.520874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8188cc5-74cc-4a83-bffc-0ff381371492 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:45.522313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 649/1315] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:25:45.527826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399d5c9-70e6-4619-93cf-341340d08d2b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:45.528244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399d5c9-70e6-4619-93cf-341340d08d2b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:45.528413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399d5c9-70e6-4619-93cf-341340d08d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:45.591941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9399d5c9-70e6-4619-93cf-341340d08d2b None None] MFA Rules not processed for user `f2817f9598a341b1bc87577b4bc383f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:25:45.593149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9399d5c9-70e6-4619-93cf-341340d08d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:45.645317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:45.689467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 667/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:25:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:25:45.730317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8188cc5-74cc-4a83-bffc-0ff381371492 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:25:45.731880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f8188cc5-74cc-4a83-bffc-0ff381371492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:45.741556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:45.834152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8188cc5-74cc-4a83-bffc-0ff381371492 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:25:45.905850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780ecc8-5dd6-48ef-9957-7ec02a994fe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:25:45.906492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780ecc8-5dd6-48ef-9957-7ec02a994fe7 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:45.906778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780ecc8-5dd6-48ef-9957-7ec02a994fe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:45.907055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780ecc8-5dd6-48ef-9957-7ec02a994fe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:45.908733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5780ecc8-5dd6-48ef-9957-7ec02a994fe7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:45.971744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:46.018708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780ecc8-5dd6-48ef-9957-7ec02a994fe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:25:46.020146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780ecc8-5dd6-48ef-9957-7ec02a994fe7 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:25:46.038602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 650/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:25:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:25:48.656287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878875c-c7f6-4e84-a6f7-102c56819d2c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:48.656892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878875c-c7f6-4e84-a6f7-102c56819d2c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:48.656892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878875c-c7f6-4e84-a6f7-102c56819d2c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:48.658238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 668/1318] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:25:48.663409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e53513-10b4-470e-825c-f36a1eb7a4cc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:48.663876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e53513-10b4-470e-825c-f36a1eb7a4cc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:48.663988 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e53513-10b4-470e-825c-f36a1eb7a4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:48.728808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-51e53513-10b4-470e-825c-f36a1eb7a4cc None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:25:48.730008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-51e53513-10b4-470e-825c-f36a1eb7a4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:48.826351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 651/1319] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:25:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:25:48.880969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c05791-69cb-40af-b7a8-b1a2ceb58ba8 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:48.881253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c05791-69cb-40af-b7a8-b1a2ceb58ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:48.881530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c05791-69cb-40af-b7a8-b1a2ceb58ba8 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:48.883882 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 669/1320] 10.222.0.57 () {54 vars in 965 bytes} [Thu Jul 21 17:25:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:25:48.889785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4541b42-c844-483a-8e2f-4d70aad9acfb None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:48.890076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4541b42-c844-483a-8e2f-4d70aad9acfb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:48.890357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4541b42-c844-483a-8e2f-4d70aad9acfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:48.911335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:48.967792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a4541b42-c844-483a-8e2f-4d70aad9acfb None None] MFA Rules not processed for user `b0edda384caf46819169b69b858afc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:25:48.969130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a4541b42-c844-483a-8e2f-4d70aad9acfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:49.045127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 652/1321] 10.222.0.57 () {58 vars in 1062 bytes} [Thu Jul 21 17:25:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:25:49.054202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c05791-69cb-40af-b7a8-b1a2ceb58ba8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:25:49.056415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b8c05791-69cb-40af-b7a8-b1a2ceb58ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:49.159298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c05791-69cb-40af-b7a8-b1a2ceb58ba8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:25:49.233974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38324eb-e23e-4d4c-824e-df47c74607a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:25:49.234551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38324eb-e23e-4d4c-824e-df47c74607a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:49.234838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38324eb-e23e-4d4c-824e-df47c74607a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:49.235118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38324eb-e23e-4d4c-824e-df47c74607a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:49.236790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c38324eb-e23e-4d4c-824e-df47c74607a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:49.246051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:49.348850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38324eb-e23e-4d4c-824e-df47c74607a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:25:49.350287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38324eb-e23e-4d4c-824e-df47c74607a4 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:25:49.368916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 670/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:25:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:25:52.539562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db410dd6-b891-49e0-a135-5655db8b5895 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:52.540104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db410dd6-b891-49e0-a135-5655db8b5895 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:52.540104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db410dd6-b891-49e0-a135-5655db8b5895 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:52.541603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 653/1323] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:25:52.547083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df988e85-808d-4a47-b4fa-60d3850201fd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:52.547531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df988e85-808d-4a47-b4fa-60d3850201fd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:52.547771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df988e85-808d-4a47-b4fa-60d3850201fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:52.614032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-df988e85-808d-4a47-b4fa-60d3850201fd None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:25:52.615233 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-df988e85-808d-4a47-b4fa-60d3850201fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:52.674794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:52.720122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 671/1324] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:25:52.758126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db410dd6-b891-49e0-a135-5655db8b5895 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:25:52.759810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-db410dd6-b891-49e0-a135-5655db8b5895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:52.871309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db410dd6-b891-49e0-a135-5655db8b5895 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:25:52.883792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:52.951126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591e547-2bec-4d5f-938b-52a23e5dd651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:25:52.951780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591e547-2bec-4d5f-938b-52a23e5dd651 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:52.952062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591e547-2bec-4d5f-938b-52a23e5dd651 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:52.952363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591e547-2bec-4d5f-938b-52a23e5dd651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:52.954359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8591e547-2bec-4d5f-938b-52a23e5dd651 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:53.062995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591e547-2bec-4d5f-938b-52a23e5dd651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:25:53.064469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591e547-2bec-4d5f-938b-52a23e5dd651 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:25:53.082964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 654/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:25:55.176994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf360ca-6fa7-4649-96ba-3c01b26cdda9 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:55.177491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf360ca-6fa7-4649-96ba-3c01b26cdda9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:55.177603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf360ca-6fa7-4649-96ba-3c01b26cdda9 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:55.178963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 672/1326] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:25:55.184371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184134a5-1d63-4f66-b915-86dda6513140 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:55.184797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184134a5-1d63-4f66-b915-86dda6513140 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:55.184941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184134a5-1d63-4f66-b915-86dda6513140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:55.235406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:55.256101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-184134a5-1d63-4f66-b915-86dda6513140 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:25:55.258043 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-184134a5-1d63-4f66-b915-86dda6513140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:55.354271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 655/1327] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:25:55.391053 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf360ca-6fa7-4649-96ba-3c01b26cdda9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:25:55.392653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bcf360ca-6fa7-4649-96ba-3c01b26cdda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:55.494068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf360ca-6fa7-4649-96ba-3c01b26cdda9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:25:55.571741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5585d-69b0-4c1f-ac7e-ebcf1974d05c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:25:55.572428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5585d-69b0-4c1f-ac7e-ebcf1974d05c service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:55.572707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5585d-69b0-4c1f-ac7e-ebcf1974d05c service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:55.572977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5585d-69b0-4c1f-ac7e-ebcf1974d05c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:55.574730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-21f5585d-69b0-4c1f-ac7e-ebcf1974d05c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:55.593428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:55.687235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5585d-69b0-4c1f-ac7e-ebcf1974d05c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:25:55.688714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5585d-69b0-4c1f-ac7e-ebcf1974d05c service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:25:55.707414 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 673/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:25:57.795337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cf7ca-25b7-4379-8e1a-9c17fce08dca None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:57.795900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cf7ca-25b7-4379-8e1a-9c17fce08dca None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:57.795900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cf7ca-25b7-4379-8e1a-9c17fce08dca None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:57.797285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 656/1329] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:25:57.802375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad317c8-cad0-42fc-bf38-6f3ad18b3297 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:57.802803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad317c8-cad0-42fc-bf38-6f3ad18b3297 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:57.803047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad317c8-cad0-42fc-bf38-6f3ad18b3297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:57.866774 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-aad317c8-cad0-42fc-bf38-6f3ad18b3297 None None] MFA Rules not processed for user `f2817f9598a341b1bc87577b4bc383f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:25:57.867966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-aad317c8-cad0-42fc-bf38-6f3ad18b3297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:57.937852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:57.970735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 674/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:25:58.009312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cf7ca-25b7-4379-8e1a-9c17fce08dca None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:25:58.010890 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ad8cf7ca-25b7-4379-8e1a-9c17fce08dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:58.115871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cf7ca-25b7-4379-8e1a-9c17fce08dca None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:25:58.136466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:25:58.190328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc3880-bf19-4343-8eb8-7dd668da38a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:25:58.190898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc3880-bf19-4343-8eb8-7dd668da38a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:25:58.191177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc3880-bf19-4343-8eb8-7dd668da38a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:25:58.191454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc3880-bf19-4343-8eb8-7dd668da38a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:25:58.193153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2efc3880-bf19-4343-8eb8-7dd668da38a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:25:58.306087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc3880-bf19-4343-8eb8-7dd668da38a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:25:58.307515 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc3880-bf19-4343-8eb8-7dd668da38a7 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:25:58.325320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 657/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:25:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:00.618684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c0cb5-fb58-4c48-9da0-a5fcc1c6e95c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:00.619280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c0cb5-fb58-4c48-9da0-a5fcc1c6e95c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:00.619280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c0cb5-fb58-4c48-9da0-a5fcc1c6e95c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:00.620642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 675/1332] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:00.625711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b373701-e769-4249-87de-8a915563177f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:00.626158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b373701-e769-4249-87de-8a915563177f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:00.626263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b373701-e769-4249-87de-8a915563177f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:00.687317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:00.690391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0b373701-e769-4249-87de-8a915563177f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:00.692213 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0b373701-e769-4249-87de-8a915563177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:00.786062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 658/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:00.822584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c0cb5-fb58-4c48-9da0-a5fcc1c6e95c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:00.824147 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e64c0cb5-fb58-4c48-9da0-a5fcc1c6e95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:00.925274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c0cb5-fb58-4c48-9da0-a5fcc1c6e95c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:00.998953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf16654-72fc-4082-8374-bb128061abab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:00.999597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf16654-72fc-4082-8374-bb128061abab service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:00.999883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf16654-72fc-4082-8374-bb128061abab service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:01.000160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf16654-72fc-4082-8374-bb128061abab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:01.001822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fcf16654-72fc-4082-8374-bb128061abab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:01.028236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:01.108434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf16654-72fc-4082-8374-bb128061abab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:01.109880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf16654-72fc-4082-8374-bb128061abab service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:01.127928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 676/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:03.206892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0defdc5-4349-4fee-beb8-bbfa02aa6beb None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:03.207382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0defdc5-4349-4fee-beb8-bbfa02aa6beb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:03.207382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0defdc5-4349-4fee-beb8-bbfa02aa6beb None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:03.208779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 659/1335] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:03.214408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa354e8-08cb-4428-9bb7-330faf3e3856 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:03.214824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa354e8-08cb-4428-9bb7-330faf3e3856 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:03.214975 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa354e8-08cb-4428-9bb7-330faf3e3856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:03.280319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7aa354e8-08cb-4428-9bb7-330faf3e3856 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:03.281545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7aa354e8-08cb-4428-9bb7-330faf3e3856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:03.357572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:03.380002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 677/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:03.421475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0defdc5-4349-4fee-beb8-bbfa02aa6beb None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:03.423004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f0defdc5-4349-4fee-beb8-bbfa02aa6beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:03.534814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0defdc5-4349-4fee-beb8-bbfa02aa6beb None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:03.563986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:03.612963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ecc77-afee-4bce-9ad7-6f41e97ec6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:03.613620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ecc77-afee-4bce-9ad7-6f41e97ec6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:03.613905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ecc77-afee-4bce-9ad7-6f41e97ec6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:03.614185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ecc77-afee-4bce-9ad7-6f41e97ec6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:03.616523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ed0ecc77-afee-4bce-9ad7-6f41e97ec6d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:03.728305 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ecc77-afee-4bce-9ad7-6f41e97ec6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:03.729757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ecc77-afee-4bce-9ad7-6f41e97ec6d1 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:03.748104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 660/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:06.111553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e30bc-31c5-4a93-9a3f-82358d74793c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:06.112103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e30bc-31c5-4a93-9a3f-82358d74793c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:06.112103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e30bc-31c5-4a93-9a3f-82358d74793c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:06.113514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 678/1338] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:06.119234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973ccda-18b8-49ac-8942-568cc1236429 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:06.119671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973ccda-18b8-49ac-8942-568cc1236429 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:06.119779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973ccda-18b8-49ac-8942-568cc1236429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:06.186070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4973ccda-18b8-49ac-8942-568cc1236429 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:06.187205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4973ccda-18b8-49ac-8942-568cc1236429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:06.197578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:06.286808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 661/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:06.325042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e30bc-31c5-4a93-9a3f-82358d74793c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:06.326561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-757e30bc-31c5-4a93-9a3f-82358d74793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:06.427586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e30bc-31c5-4a93-9a3f-82358d74793c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:06.499749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e82a3-0843-4548-a9bd-2256ba06eb00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:06.500337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e82a3-0843-4548-a9bd-2256ba06eb00 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:06.500626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e82a3-0843-4548-a9bd-2256ba06eb00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:06.500902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e82a3-0843-4548-a9bd-2256ba06eb00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:06.502536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c80e82a3-0843-4548-a9bd-2256ba06eb00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:06.536669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:06.609680 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e82a3-0843-4548-a9bd-2256ba06eb00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:06.611109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e82a3-0843-4548-a9bd-2256ba06eb00 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:06.629249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 679/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:10.781638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143c91e-70ba-4aba-bbf8-4850176fb81b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:10.782146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143c91e-70ba-4aba-bbf8-4850176fb81b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:10.782146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143c91e-70ba-4aba-bbf8-4850176fb81b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:10.783491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 662/1341] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:10.789427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812daf7c-c7ac-426c-a7b7-e6e2f5cce03a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:10.789834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812daf7c-c7ac-426c-a7b7-e6e2f5cce03a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:10.790018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812daf7c-c7ac-426c-a7b7-e6e2f5cce03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:10.855877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-812daf7c-c7ac-426c-a7b7-e6e2f5cce03a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:10.857018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-812daf7c-c7ac-426c-a7b7-e6e2f5cce03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:10.946992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:10.960248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 680/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:26:10.999882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143c91e-70ba-4aba-bbf8-4850176fb81b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:11.001542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6143c91e-70ba-4aba-bbf8-4850176fb81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:11.109705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143c91e-70ba-4aba-bbf8-4850176fb81b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:11.148997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:11.186256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee60510-ac3c-48b6-9a4c-e67f68001467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:11.186917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee60510-ac3c-48b6-9a4c-e67f68001467 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:11.187204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee60510-ac3c-48b6-9a4c-e67f68001467 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:11.187481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee60510-ac3c-48b6-9a4c-e67f68001467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:11.189447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0ee60510-ac3c-48b6-9a4c-e67f68001467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:11.298060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee60510-ac3c-48b6-9a4c-e67f68001467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:11.299414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee60510-ac3c-48b6-9a4c-e67f68001467 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:11.317724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 663/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:13.938423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c5d2f-012e-4b78-8558-882457420d35 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:13.939161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c5d2f-012e-4b78-8558-882457420d35 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:13.939370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c5d2f-012e-4b78-8558-882457420d35 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:13.940722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 681/1344] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:13.946563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7039431-bd9e-4464-9eca-9bacd07c211a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:13.947096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7039431-bd9e-4464-9eca-9bacd07c211a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:13.947096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7039431-bd9e-4464-9eca-9bacd07c211a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:14.018003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a7039431-bd9e-4464-9eca-9bacd07c211a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:14.019233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a7039431-bd9e-4464-9eca-9bacd07c211a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:14.039075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:14.123433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 664/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:14.161320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c5d2f-012e-4b78-8558-882457420d35 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:14.163632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-409c5d2f-012e-4b78-8558-882457420d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:14.269080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c5d2f-012e-4b78-8558-882457420d35 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:14.343761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74785598-be6e-4c5e-ae3d-5f8be73572b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:14.344442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74785598-be6e-4c5e-ae3d-5f8be73572b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:14.344726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74785598-be6e-4c5e-ae3d-5f8be73572b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:14.344998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74785598-be6e-4c5e-ae3d-5f8be73572b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:14.346709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-74785598-be6e-4c5e-ae3d-5f8be73572b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:14.391293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:14.459898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74785598-be6e-4c5e-ae3d-5f8be73572b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:14.461392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74785598-be6e-4c5e-ae3d-5f8be73572b6 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:14.479577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 682/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:18.575701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fadb24-1531-48d0-9db4-e889f1f736e0 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:18.576243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fadb24-1531-48d0-9db4-e889f1f736e0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:18.576243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fadb24-1531-48d0-9db4-e889f1f736e0 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:18.577645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 665/1347] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:18.582930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c51a4f-a4be-411b-ba47-a72973ee9acc None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:18.583398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c51a4f-a4be-411b-ba47-a72973ee9acc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:18.583686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c51a4f-a4be-411b-ba47-a72973ee9acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:18.652331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a6c51a4f-a4be-411b-ba47-a72973ee9acc None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:18.653551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a6c51a4f-a4be-411b-ba47-a72973ee9acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:18.768756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:18.772644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 683/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:18.816344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fadb24-1531-48d0-9db4-e889f1f736e0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:18.817914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-48fadb24-1531-48d0-9db4-e889f1f736e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:18.928938 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fadb24-1531-48d0-9db4-e889f1f736e0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:18.980578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:19.007965 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3413c2-61c7-4fec-8717-4555ed5b4378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:19.008652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3413c2-61c7-4fec-8717-4555ed5b4378 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:19.008938 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3413c2-61c7-4fec-8717-4555ed5b4378 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:19.009223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3413c2-61c7-4fec-8717-4555ed5b4378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:19.010902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eb3413c2-61c7-4fec-8717-4555ed5b4378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:19.125945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3413c2-61c7-4fec-8717-4555ed5b4378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:19.127385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3413c2-61c7-4fec-8717-4555ed5b4378 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:19.147166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 666/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:24.058797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa7af9-025d-49e0-b6fb-1befb3f0414b None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:24.059316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa7af9-025d-49e0-b6fb-1befb3f0414b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:24.059395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa7af9-025d-49e0-b6fb-1befb3f0414b None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:24.061461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 684/1350] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 17:26:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:24.068412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6c25e-1e8f-4ff3-90d4-e5420dd412b6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:24.068854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6c25e-1e8f-4ff3-90d4-e5420dd412b6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:24.068984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6c25e-1e8f-4ff3-90d4-e5420dd412b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:24.194415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-19b6c25e-1e8f-4ff3-90d4-e5420dd412b6 None None] MFA Rules not processed for user `901cf70064c34b63a0eda979a5ce77e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:24.195592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-19b6c25e-1e8f-4ff3-90d4-e5420dd412b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:24.221815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:24.312181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 667/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:28.421166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf45e3b-796a-43d2-9a3f-dba5380e7fe0 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:28.422085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf45e3b-796a-43d2-9a3f-dba5380e7fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:28.422378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf45e3b-796a-43d2-9a3f-dba5380e7fe0 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:28.423957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 685/1352] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 17:26:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:28.429233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42515c0-d6a1-4186-a5e7-c441e0609115 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:28.429671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42515c0-d6a1-4186-a5e7-c441e0609115 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:28.429794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42515c0-d6a1-4186-a5e7-c441e0609115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:28.519253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f42515c0-d6a1-4186-a5e7-c441e0609115 None None] MFA Rules not processed for user `901cf70064c34b63a0eda979a5ce77e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:28.521143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f42515c0-d6a1-4186-a5e7-c441e0609115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:28.602511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:28.640686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 668/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 17:26:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:30.119557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d0c5c-9312-420b-ac28-2a206885b5ae None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:30.120178 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d0c5c-9312-420b-ac28-2a206885b5ae None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:30.120178 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d0c5c-9312-420b-ac28-2a206885b5ae None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:30.121670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 686/1354] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 17:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:30.126678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412bae6-3ac7-4270-94de-392cfabc530f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:30.127138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412bae6-3ac7-4270-94de-392cfabc530f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:30.127243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412bae6-3ac7-4270-94de-392cfabc530f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:30.203130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7412bae6-3ac7-4270-94de-392cfabc530f None None] MFA Rules not processed for user `901cf70064c34b63a0eda979a5ce77e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:30.204264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7412bae6-3ac7-4270-94de-392cfabc530f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:30.311999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 669/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 17:26:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:26:33.736170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144f5bc-7936-404a-a6a6-49d6ece0091a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:33.737050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144f5bc-7936-404a-a6a6-49d6ece0091a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:33.737503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144f5bc-7936-404a-a6a6-49d6ece0091a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:33.739855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 687/1356] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:33.745568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6e23-46a3-43bd-9332-3e11bc816aeb None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:33.746025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6e23-46a3-43bd-9332-3e11bc816aeb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:33.746576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6e23-46a3-43bd-9332-3e11bc816aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:33.790607 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:33.826730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-13ba6e23-46a3-43bd-9332-3e11bc816aeb None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:33.827930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-13ba6e23-46a3-43bd-9332-3e11bc816aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:33.932456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 670/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:33.984557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fb9f0-ddb9-4e56-b9cd-c647d39242f8 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:33.985620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fb9f0-ddb9-4e56-b9cd-c647d39242f8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:33.986211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fb9f0-ddb9-4e56-b9cd-c647d39242f8 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:33.987824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 688/1358] 10.222.0.57 () {54 vars in 962 bytes} [Thu Jul 21 17:26:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:34.002053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cd778-c557-4b93-987b-b9ede5184910 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:34.002421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cd778-c557-4b93-987b-b9ede5184910 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:34.002869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cd778-c557-4b93-987b-b9ede5184910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:34.084539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-431cd778-c557-4b93-987b-b9ede5184910 None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:34.085737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-431cd778-c557-4b93-987b-b9ede5184910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:34.137761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:34.214469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 671/1359] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Jul 21 17:26:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:34.222372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fb9f0-ddb9-4e56-b9cd-c647d39242f8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:34.223923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-051fb9f0-ddb9-4e56-b9cd-c647d39242f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:34.376749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fb9f0-ddb9-4e56-b9cd-c647d39242f8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:34.454450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:34.497825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcb98c-57ac-47c3-878e-f879ff287cd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:34.498395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcb98c-57ac-47c3-878e-f879ff287cd8 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:34.498683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcb98c-57ac-47c3-878e-f879ff287cd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:34.498959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcb98c-57ac-47c3-878e-f879ff287cd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:34.500618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e5fcb98c-57ac-47c3-878e-f879ff287cd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:34.600925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcb98c-57ac-47c3-878e-f879ff287cd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:34.602518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcb98c-57ac-47c3-878e-f879ff287cd8 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:34.621641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 689/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:36.890218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fa87d-4fcd-4549-b246-a30a937ff67e None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:36.890756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fa87d-4fcd-4549-b246-a30a937ff67e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:36.890756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fa87d-4fcd-4549-b246-a30a937ff67e None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:36.892228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 672/1361] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:36.897780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3f6a2-a32d-438b-9048-6717e85dc9e3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:36.898179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3f6a2-a32d-438b-9048-6717e85dc9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:36.898323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3f6a2-a32d-438b-9048-6717e85dc9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:36.973624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5bc3f6a2-a32d-438b-9048-6717e85dc9e3 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:36.974837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5bc3f6a2-a32d-438b-9048-6717e85dc9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:37.014681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:37.088342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 690/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:37.137743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fa87d-4fcd-4549-b246-a30a937ff67e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:37.140315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-774fa87d-4fcd-4549-b246-a30a937ff67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:37.353126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fa87d-4fcd-4549-b246-a30a937ff67e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:37.372650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:37.499259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b5ff5-2069-4e0b-a61b-018d6ddff3e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:37.499919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b5ff5-2069-4e0b-a61b-018d6ddff3e5 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:37.500204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b5ff5-2069-4e0b-a61b-018d6ddff3e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:37.500513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b5ff5-2069-4e0b-a61b-018d6ddff3e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:37.502429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2b0b5ff5-2069-4e0b-a61b-018d6ddff3e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:37.604173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b5ff5-2069-4e0b-a61b-018d6ddff3e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:37.606044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b5ff5-2069-4e0b-a61b-018d6ddff3e5 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:37.627367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 673/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:26:40.028100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dbf0c-f73a-4e4a-944e-c1e33650d312 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:40.028688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dbf0c-f73a-4e4a-944e-c1e33650d312 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:40.028688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dbf0c-f73a-4e4a-944e-c1e33650d312 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:40.030113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 691/1364] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:40.036418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222e301-27a5-4fb3-8232-f6aab10ada62 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:40.036900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222e301-27a5-4fb3-8232-f6aab10ada62 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:40.036982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222e301-27a5-4fb3-8232-f6aab10ada62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:40.052353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:40.108165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6222e301-27a5-4fb3-8232-f6aab10ada62 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:40.109553 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6222e301-27a5-4fb3-8232-f6aab10ada62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:40.228895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 674/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:40.276994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb914af6-97ab-44bb-9d28-a786878a2487 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:40.277318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb914af6-97ab-44bb-9d28-a786878a2487 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:40.277773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb914af6-97ab-44bb-9d28-a786878a2487 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:40.279100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 692/1366] 10.222.0.57 () {54 vars in 962 bytes} [Thu Jul 21 17:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:40.287372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a9d1-030d-4ea4-a456-04c620d161aa None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:40.288263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a9d1-030d-4ea4-a456-04c620d161aa None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:40.288874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a9d1-030d-4ea4-a456-04c620d161aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:40.370814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b166a9d1-030d-4ea4-a456-04c620d161aa None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:40.372023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b166a9d1-030d-4ea4-a456-04c620d161aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:40.408885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:40.504228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 675/1367] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Jul 21 17:26:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:40.512481 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb914af6-97ab-44bb-9d28-a786878a2487 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:40.514001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fb914af6-97ab-44bb-9d28-a786878a2487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:40.678973 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb914af6-97ab-44bb-9d28-a786878a2487 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:40.701307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:40.803125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d748292-4f2c-414f-bdcb-f08363eefe22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:40.804040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d748292-4f2c-414f-bdcb-f08363eefe22 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:40.804040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d748292-4f2c-414f-bdcb-f08363eefe22 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:40.804247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d748292-4f2c-414f-bdcb-f08363eefe22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:40.805906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2d748292-4f2c-414f-bdcb-f08363eefe22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:40.902313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d748292-4f2c-414f-bdcb-f08363eefe22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:40.903933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d748292-4f2c-414f-bdcb-f08363eefe22 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:40.939093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 693/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:41.908833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c09f22-37d7-475d-90fb-e4effec5beda None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:41.909395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c09f22-37d7-475d-90fb-e4effec5beda None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:41.909395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c09f22-37d7-475d-90fb-e4effec5beda None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:41.911094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 676/1369] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:41.916490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f476eb6-3701-4101-bdf1-24d4addf8849 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:41.916964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f476eb6-3701-4101-bdf1-24d4addf8849 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:41.917049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f476eb6-3701-4101-bdf1-24d4addf8849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:41.949734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:41.985748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1f476eb6-3701-4101-bdf1-24d4addf8849 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:41.986922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1f476eb6-3701-4101-bdf1-24d4addf8849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:42.088504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 694/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:42.432350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150141c-d3dc-4b76-8df6-dd8ade6adf39 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:42.432350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150141c-d3dc-4b76-8df6-dd8ade6adf39 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:42.432350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150141c-d3dc-4b76-8df6-dd8ade6adf39 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:42.434079 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 677/1371] 10.222.0.57 () {54 vars in 964 bytes} [Thu Jul 21 17:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:42.438663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77876a31-3052-47fe-b0e6-11eb7d4b20f5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:42.438950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77876a31-3052-47fe-b0e6-11eb7d4b20f5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:42.439224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77876a31-3052-47fe-b0e6-11eb7d4b20f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:42.521677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-77876a31-3052-47fe-b0e6-11eb7d4b20f5 None None] MFA Rules not processed for user `9bbddf0ea88049aa9bb6a257e1f01ffd`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:42.522870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-77876a31-3052-47fe-b0e6-11eb7d4b20f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:42.573090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:42.605201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 695/1372] 10.222.0.57 () {58 vars in 1061 bytes} [Thu Jul 21 17:26:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:42.612516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150141c-d3dc-4b76-8df6-dd8ade6adf39 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:42.614167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3150141c-d3dc-4b76-8df6-dd8ade6adf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:42.732866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150141c-d3dc-4b76-8df6-dd8ade6adf39 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:42.775383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:42.811298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1311831-9aa4-4837-852d-214133e7d340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:42.812320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1311831-9aa4-4837-852d-214133e7d340 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:42.814857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1311831-9aa4-4837-852d-214133e7d340 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:42.814857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1311831-9aa4-4837-852d-214133e7d340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:42.815085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c1311831-9aa4-4837-852d-214133e7d340 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:42.937290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1311831-9aa4-4837-852d-214133e7d340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:42.939184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1311831-9aa4-4837-852d-214133e7d340 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:42.961600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 678/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:43.059203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04993888-fca3-4e79-b023-aa09f6e9f854 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:43.059925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04993888-fca3-4e79-b023-aa09f6e9f854 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:43.059925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04993888-fca3-4e79-b023-aa09f6e9f854 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:43.066951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 696/1374] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:43.067401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151e82f-4105-4ff2-beb9-a7912d949443 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:43.067724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151e82f-4105-4ff2-beb9-a7912d949443 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:43.067896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151e82f-4105-4ff2-beb9-a7912d949443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:43.144315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b151e82f-4105-4ff2-beb9-a7912d949443 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:43.145792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b151e82f-4105-4ff2-beb9-a7912d949443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:43.167161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:43.246275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 679/1375] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:43.290274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04993888-fca3-4e79-b023-aa09f6e9f854 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:43.291738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-04993888-fca3-4e79-b023-aa09f6e9f854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:43.440002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04993888-fca3-4e79-b023-aa09f6e9f854 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:43.490695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:43.552523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c38be2-718c-4d6e-8936-65b11915862b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:43.553169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38be2-718c-4d6e-8936-65b11915862b service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:43.553450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38be2-718c-4d6e-8936-65b11915862b service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:43.553723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38be2-718c-4d6e-8936-65b11915862b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:43.555350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c1c38be2-718c-4d6e-8936-65b11915862b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:43.653258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38be2-718c-4d6e-8936-65b11915862b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:43.654853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38be2-718c-4d6e-8936-65b11915862b service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:43.673294 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 697/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:45.788925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916667d7-d9e4-4354-9b5c-337b338bf192 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:45.789428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916667d7-d9e4-4354-9b5c-337b338bf192 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:45.789547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916667d7-d9e4-4354-9b5c-337b338bf192 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:45.790879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 680/1377] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:45.796119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81fe0f-82df-4b22-b1b6-a8036115cd36 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:45.796593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81fe0f-82df-4b22-b1b6-a8036115cd36 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:45.796722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81fe0f-82df-4b22-b1b6-a8036115cd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:45.869347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0d81fe0f-82df-4b22-b1b6-a8036115cd36 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:45.870559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0d81fe0f-82df-4b22-b1b6-a8036115cd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:45.877276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:45.999040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 698/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:46.052244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916667d7-d9e4-4354-9b5c-337b338bf192 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:46.054042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-916667d7-d9e4-4354-9b5c-337b338bf192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:46.203323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916667d7-d9e4-4354-9b5c-337b338bf192 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:46.206961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:46.318754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4e8b-5112-450a-b80c-8d88deb95a79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:46.319323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4e8b-5112-450a-b80c-8d88deb95a79 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:46.319605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4e8b-5112-450a-b80c-8d88deb95a79 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:46.319881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4e8b-5112-450a-b80c-8d88deb95a79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:46.321572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6c9a4e8b-5112-450a-b80c-8d88deb95a79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:46.423314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4e8b-5112-450a-b80c-8d88deb95a79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:46.428809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4e8b-5112-450a-b80c-8d88deb95a79 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:46.455366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 681/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:48.602537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4656cc1-8ad9-46be-9a3a-7170dcfaf910 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:48.603092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4656cc1-8ad9-46be-9a3a-7170dcfaf910 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:48.603092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4656cc1-8ad9-46be-9a3a-7170dcfaf910 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:48.604578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 699/1380] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:48.609918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde4b04-5250-4335-b122-cc830ec32164 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:48.610329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde4b04-5250-4335-b122-cc830ec32164 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:48.610470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde4b04-5250-4335-b122-cc830ec32164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:48.621344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:48.685215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6fde4b04-5250-4335-b122-cc830ec32164 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:48.686417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6fde4b04-5250-4335-b122-cc830ec32164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:48.787552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 682/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:48.832122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4656cc1-8ad9-46be-9a3a-7170dcfaf910 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:48.834378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c4656cc1-8ad9-46be-9a3a-7170dcfaf910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:48.995094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:49.007281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4656cc1-8ad9-46be-9a3a-7170dcfaf910 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:49.138286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a019cb-35a7-449b-aea1-97e5460d6222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:49.138858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a019cb-35a7-449b-aea1-97e5460d6222 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:49.139142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a019cb-35a7-449b-aea1-97e5460d6222 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:49.139420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a019cb-35a7-449b-aea1-97e5460d6222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:49.141096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-21a019cb-35a7-449b-aea1-97e5460d6222 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:49.238967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a019cb-35a7-449b-aea1-97e5460d6222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:49.240706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a019cb-35a7-449b-aea1-97e5460d6222 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:49.251256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:49.261129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 700/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:51.594956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9160-00b9-4c68-ad0c-ddbd91e55d8c None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:51.595506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9160-00b9-4c68-ad0c-ddbd91e55d8c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:51.595506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9160-00b9-4c68-ad0c-ddbd91e55d8c None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:51.597012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 683/1383] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:51.603834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b9c8e-6222-4ef2-b003-6d3e1231ea6b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:51.604258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b9c8e-6222-4ef2-b003-6d3e1231ea6b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:51.604658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b9c8e-6222-4ef2-b003-6d3e1231ea6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:51.678880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a29b9c8e-6222-4ef2-b003-6d3e1231ea6b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:51.680176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a29b9c8e-6222-4ef2-b003-6d3e1231ea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:51.803194 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 701/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:51.845175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9160-00b9-4c68-ad0c-ddbd91e55d8c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:51.847215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-99be9160-00b9-4c68-ad0c-ddbd91e55d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:51.951172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:52.016983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9160-00b9-4c68-ad0c-ddbd91e55d8c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:52.133627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bbbd2-a8d1-482d-a07b-be1ee48b74f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:52.134292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bbbd2-a8d1-482d-a07b-be1ee48b74f0 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:52.134576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bbbd2-a8d1-482d-a07b-be1ee48b74f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:52.134857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bbbd2-a8d1-482d-a07b-be1ee48b74f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:52.136576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-779bbbd2-a8d1-482d-a07b-be1ee48b74f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:52.185761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:52.227305 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bbbd2-a8d1-482d-a07b-be1ee48b74f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:52.228949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bbbd2-a8d1-482d-a07b-be1ee48b74f0 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:52.247554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 684/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:54.287760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad54be-f540-4fbf-b6be-42ecea5fb048 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:54.288484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad54be-f540-4fbf-b6be-42ecea5fb048 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:54.288484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad54be-f540-4fbf-b6be-42ecea5fb048 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:54.289620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 702/1386] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:54.295002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf9197-d3d1-47c9-a15a-2164237622bd None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:54.295552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf9197-d3d1-47c9-a15a-2164237622bd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:54.295552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf9197-d3d1-47c9-a15a-2164237622bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:54.369472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-11bf9197-d3d1-47c9-a15a-2164237622bd None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:54.371116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-11bf9197-d3d1-47c9-a15a-2164237622bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:54.487328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 685/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:26:54.536941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad54be-f540-4fbf-b6be-42ecea5fb048 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:54.538502 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-77ad54be-f540-4fbf-b6be-42ecea5fb048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:54.620552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:54.698850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad54be-f540-4fbf-b6be-42ecea5fb048 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:54.826127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8376fa-773b-4874-9af2-dc8d78226b27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:54.826932 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8376fa-773b-4874-9af2-dc8d78226b27 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:54.827286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8376fa-773b-4874-9af2-dc8d78226b27 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:54.827643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8376fa-773b-4874-9af2-dc8d78226b27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:54.829884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ad8376fa-773b-4874-9af2-dc8d78226b27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:54.872104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:54.930014 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8376fa-773b-4874-9af2-dc8d78226b27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:54.931618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8376fa-773b-4874-9af2-dc8d78226b27 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:54.950533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 703/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:26:57.039851 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb718-23bc-420e-9e39-6a0e69b681d3 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:57.040458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb718-23bc-420e-9e39-6a0e69b681d3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:57.040458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb718-23bc-420e-9e39-6a0e69b681d3 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:57.041825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 686/1389] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:57.047062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc33d17-6393-4df4-87ef-9414842fbf21 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:57.047487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc33d17-6393-4df4-87ef-9414842fbf21 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:57.047619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc33d17-6393-4df4-87ef-9414842fbf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:57.113426 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bbc33d17-6393-4df4-87ef-9414842fbf21 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:57.114622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bbc33d17-6393-4df4-87ef-9414842fbf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:57.216415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 704/1390] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:57.261404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdb718-23bc-420e-9e39-6a0e69b681d3 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:57.262965 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ffbdb718-23bc-420e-9e39-6a0e69b681d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:57.293670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:57.408089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdb718-23bc-420e-9e39-6a0e69b681d3 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:57.510511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:57.522700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1bff8-2b00-43e4-9965-834bab0b8747 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:57.523353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1bff8-2b00-43e4-9965-834bab0b8747 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:57.523639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1bff8-2b00-43e4-9965-834bab0b8747 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:57.523912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1bff8-2b00-43e4-9965-834bab0b8747 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:57.525581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1cb1bff8-2b00-43e4-9965-834bab0b8747 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:57.614171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1bff8-2b00-43e4-9965-834bab0b8747 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:57.615783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1bff8-2b00-43e4-9965-834bab0b8747 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:57.634451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 687/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:58.061797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7625331-205c-4c17-b9ee-aac540f32322 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:58.062945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7625331-205c-4c17-b9ee-aac540f32322 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:58.063226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7625331-205c-4c17-b9ee-aac540f32322 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:58.064642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 705/1392] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:58.070027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdcacf-846a-491c-86c5-b246221ca736 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:58.070472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdcacf-846a-491c-86c5-b246221ca736 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:58.070595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdcacf-846a-491c-86c5-b246221ca736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:58.136672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a3fdcacf-846a-491c-86c5-b246221ca736 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:58.137866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a3fdcacf-846a-491c-86c5-b246221ca736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:58.198994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:58.237813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 688/1393] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:58.293386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d17d4b-5e49-4d92-a2fd-098f444faad0 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:58.293673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d17d4b-5e49-4d92-a2fd-098f444faad0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:58.293948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d17d4b-5e49-4d92-a2fd-098f444faad0 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:58.295347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 706/1394] 10.222.0.57 () {54 vars in 966 bytes} [Thu Jul 21 17:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:58.300567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d2164-c491-451a-b023-aa4987717aff None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:58.300865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d2164-c491-451a-b023-aa4987717aff None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:58.301171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d2164-c491-451a-b023-aa4987717aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:58.376046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b62d2164-c491-451a-b023-aa4987717aff None None] MFA Rules not processed for user `9e07c59babad479cadbf74f15ee5eea9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:58.377612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b62d2164-c491-451a-b023-aa4987717aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:58.453704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 689/1395] 10.222.0.57 () {58 vars in 1063 bytes} [Thu Jul 21 17:26:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:58.462082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d17d4b-5e49-4d92-a2fd-098f444faad0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:58.463601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c7d17d4b-5e49-4d92-a2fd-098f444faad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:58.477111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:58.569507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d17d4b-5e49-4d92-a2fd-098f444faad0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:26:58.641917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a3b3a-d5ec-43d7-8f65-bde890ebe8ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e07c59babad479cadbf74f15ee5eea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:26:58.642560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a3b3a-d5ec-43d7-8f65-bde890ebe8ae service glance] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:58.642867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a3b3a-d5ec-43d7-8f65-bde890ebe8ae service glance] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:58.643787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a3b3a-d5ec-43d7-8f65-bde890ebe8ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:58.645458 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c83a3b3a-d5ec-43d7-8f65-bde890ebe8ae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:58.713286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:58.755348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a3b3a-d5ec-43d7-8f65-bde890ebe8ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:26:58.756835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a3b3a-d5ec-43d7-8f65-bde890ebe8ae service glance] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:26:58.783112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 707/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:26:59.517540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578437b-a5ec-479b-b10a-bb5867102801 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:59.518522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578437b-a5ec-479b-b10a-bb5867102801 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:59.518917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578437b-a5ec-479b-b10a-bb5867102801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:59.587284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2578437b-a5ec-479b-b10a-bb5867102801 None None] MFA Rules not processed for user `75f11eaa4b504d0aa4018fb4ffbbd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:59.588545 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2578437b-a5ec-479b-b10a-bb5867102801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:59.596686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:59.676775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 690/1397] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 17:26:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:59.697348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e3e85-2d2e-4874-a189-1ab52e00b4f6 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:59.698092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e3e85-2d2e-4874-a189-1ab52e00b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:59.698371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e3e85-2d2e-4874-a189-1ab52e00b4f6 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:59.699783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 708/1398] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:26:59.705405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4f1d5-b308-4c8d-a44c-f6359106dbca None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:59.705784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4f1d5-b308-4c8d-a44c-f6359106dbca None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:59.706067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4f1d5-b308-4c8d-a44c-f6359106dbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:59.719889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93642613-a7a9-4198-b74a-9d4c57639727 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:59.720177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93642613-a7a9-4198-b74a-9d4c57639727 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:59.720486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93642613-a7a9-4198-b74a-9d4c57639727 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:59.721886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 709/1399] 10.222.0.57 () {54 vars in 943 bytes} [Thu Jul 21 17:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:26:59.728196 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52389d5f-4f2b-4a3c-8de5-d48b92addc51 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:26:59.728514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52389d5f-4f2b-4a3c-8de5-d48b92addc51 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:26:59.728796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52389d5f-4f2b-4a3c-8de5-d48b92addc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:26:59.773813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-83b4f1d5-b308-4c8d-a44c-f6359106dbca None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:59.774962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-83b4f1d5-b308-4c8d-a44c-f6359106dbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:59.804803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-52389d5f-4f2b-4a3c-8de5-d48b92addc51 None None] MFA Rules not processed for user `8a26412cf9f64b1bbbd8e595b55bcebc`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:26:59.805983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-52389d5f-4f2b-4a3c-8de5-d48b92addc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:59.872154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 691/1400] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:26:59.913380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 710/1401] 10.222.0.57 () {58 vars in 1040 bytes} [Thu Jul 21 17:26:59 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:26:59.917689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4f1d5-b308-4c8d-a44c-f6359106dbca None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:59.918894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-83b4f1d5-b308-4c8d-a44c-f6359106dbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:26:59.921673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:59.927847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52389d5f-4f2b-4a3c-8de5-d48b92addc51 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:26:59.932635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:26:59.938628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52389d5f-4f2b-4a3c-8de5-d48b92addc51 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:00.047735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ab3ce-3e6e-45b0-a9a1-060023956d15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a26412cf9f64b1bbbd8e595b55bcebc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:00.048321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ab3ce-3e6e-45b0-a9a1-060023956d15 service swift] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:00.048624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ab3ce-3e6e-45b0-a9a1-060023956d15 service swift] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:00.048951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ab3ce-3e6e-45b0-a9a1-060023956d15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:00.049756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4f1d5-b308-4c8d-a44c-f6359106dbca None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:00.055213 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4f1ab3ce-3e6e-45b0-a9a1-060023956d15 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:00.142210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ab3ce-3e6e-45b0-a9a1-060023956d15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:00.143912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ab3ce-3e6e-45b0-a9a1-060023956d15 service swift] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:00.152219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 711/1402] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jul 21 17:26:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 17:27:00.166380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:00.169497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ded862-86ce-498a-a53b-db5099c83661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:00.170139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ded862-86ce-498a-a53b-db5099c83661 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:00.170726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ded862-86ce-498a-a53b-db5099c83661 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:00.171259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ded862-86ce-498a-a53b-db5099c83661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:00.173329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d9ded862-86ce-498a-a53b-db5099c83661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:00.280225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ded862-86ce-498a-a53b-db5099c83661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:00.281877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ded862-86ce-498a-a53b-db5099c83661 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:00.303516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 692/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:02.443646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df68a16-b94e-4b26-8845-537a2a64618a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:02.444242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df68a16-b94e-4b26-8845-537a2a64618a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:02.444242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df68a16-b94e-4b26-8845-537a2a64618a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:02.445787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 712/1404] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:02.450896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84082-6150-4f99-85a8-eedc1f3def4e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:02.451317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84082-6150-4f99-85a8-eedc1f3def4e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:02.451518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84082-6150-4f99-85a8-eedc1f3def4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:02.527914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1eb84082-6150-4f99-85a8-eedc1f3def4e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:02.529236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1eb84082-6150-4f99-85a8-eedc1f3def4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:02.605097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:02.642519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 693/1405] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:02.686670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df68a16-b94e-4b26-8845-537a2a64618a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:02.688550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1df68a16-b94e-4b26-8845-537a2a64618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:02.723501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:02.861447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df68a16-b94e-4b26-8845-537a2a64618a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:02.981479 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:02.994262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0f565-0e20-4653-8bec-0251251457ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:02.994919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0f565-0e20-4653-8bec-0251251457ab service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:02.995202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0f565-0e20-4653-8bec-0251251457ab service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:02.995479 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0f565-0e20-4653-8bec-0251251457ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:02.997286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a5d0f565-0e20-4653-8bec-0251251457ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:03.093456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0f565-0e20-4653-8bec-0251251457ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:03.095058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0f565-0e20-4653-8bec-0251251457ab service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:03.116836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 713/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:05.207099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff601c-149a-460f-bff6-2d034a7118d6 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:05.208088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff601c-149a-460f-bff6-2d034a7118d6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:05.208465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff601c-149a-460f-bff6-2d034a7118d6 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:05.212891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 694/1407] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:27:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:05.215490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b173f-3f30-4e60-b947-a210d5f83911 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:05.215940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b173f-3f30-4e60-b947-a210d5f83911 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:05.216107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b173f-3f30-4e60-b947-a210d5f83911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:05.288589 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a46b173f-3f30-4e60-b947-a210d5f83911 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:05.290099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a46b173f-3f30-4e60-b947-a210d5f83911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:05.359586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:05.402680 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 714/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:05.450002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff601c-149a-460f-bff6-2d034a7118d6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:05.451457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b0ff601c-149a-460f-bff6-2d034a7118d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:05.623960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff601c-149a-460f-bff6-2d034a7118d6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:05.690957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:05.758549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878c9e6-e425-48df-84c8-a4d7af651cf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:05.759596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878c9e6-e425-48df-84c8-a4d7af651cf5 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:05.760091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878c9e6-e425-48df-84c8-a4d7af651cf5 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:05.760410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878c9e6-e425-48df-84c8-a4d7af651cf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:05.762067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c878c9e6-e425-48df-84c8-a4d7af651cf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:05.901067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878c9e6-e425-48df-84c8-a4d7af651cf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:05.902823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878c9e6-e425-48df-84c8-a4d7af651cf5 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:05.931317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 695/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:15.399506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=14 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:15.935275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=73 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:22.691108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ff1a6-6e43-48b7-a2a7-56eaae01998e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:22.691724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ff1a6-6e43-48b7-a2a7-56eaae01998e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:22.691724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ff1a6-6e43-48b7-a2a7-56eaae01998e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:22.693791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 715/1410] 10.222.0.57 () {54 vars in 918 bytes} [Thu Jul 21 17:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:22.698971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bdb57-97d7-4281-a4eb-c14b733c883b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:22.699395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bdb57-97d7-4281-a4eb-c14b733c883b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:22.699508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bdb57-97d7-4281-a4eb-c14b733c883b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:22.773259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-207bdb57-97d7-4281-a4eb-c14b733c883b None None] MFA Rules not processed for user `901cf70064c34b63a0eda979a5ce77e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:22.774491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-207bdb57-97d7-4281-a4eb-c14b733c883b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:22.881577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 696/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Jul 21 17:27:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:22.896065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e63500-556a-4dd6-b341-cdb12a0af001 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:22.896384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e63500-556a-4dd6-b341-cdb12a0af001 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:22.896686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e63500-556a-4dd6-b341-cdb12a0af001 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:22.898036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 716/1412] 10.222.0.57 () {54 vars in 918 bytes} [Thu Jul 21 17:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:22.902632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e76d9b-ce9e-410e-a14d-c07cf5a56128 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:22.902927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e76d9b-ce9e-410e-a14d-c07cf5a56128 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:22.903207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e76d9b-ce9e-410e-a14d-c07cf5a56128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:22.974820 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:22.977540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-58e76d9b-ce9e-410e-a14d-c07cf5a56128 None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:22.979362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-58e76d9b-ce9e-410e-a14d-c07cf5a56128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:23.095945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 697/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Jul 21 17:27:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:27:25.582181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217063bb-6205-40ad-89a4-68846ac0da38 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:25.582692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217063bb-6205-40ad-89a4-68846ac0da38 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:25.582692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217063bb-6205-40ad-89a4-68846ac0da38 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:25.584050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:25.589314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89249505-24dc-4cdc-875d-c3be253a000f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:25.589759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89249505-24dc-4cdc-875d-c3be253a000f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:25.589881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89249505-24dc-4cdc-875d-c3be253a000f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:25.659007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-89249505-24dc-4cdc-875d-c3be253a000f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:25.660220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-89249505-24dc-4cdc-875d-c3be253a000f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:25.717346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:25.759775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:25.803701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217063bb-6205-40ad-89a4-68846ac0da38 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:25.805280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-217063bb-6205-40ad-89a4-68846ac0da38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:25.950276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217063bb-6205-40ad-89a4-68846ac0da38 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:26.023852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:26.064795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71952c2f-6d8f-42b4-b903-fe1c0e1be1c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:26.065446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71952c2f-6d8f-42b4-b903-fe1c0e1be1c1 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:26.065731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71952c2f-6d8f-42b4-b903-fe1c0e1be1c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:26.066008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71952c2f-6d8f-42b4-b903-fe1c0e1be1c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:26.067681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-71952c2f-6d8f-42b4-b903-fe1c0e1be1c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:26.155405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71952c2f-6d8f-42b4-b903-fe1c0e1be1c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:26.157043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71952c2f-6d8f-42b4-b903-fe1c0e1be1c1 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:26.174576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 718/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:27.095378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab2fbb-d053-4c52-9e5c-fc3480563463 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:27.095937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab2fbb-d053-4c52-9e5c-fc3480563463 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:27.095937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab2fbb-d053-4c52-9e5c-fc3480563463 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:27.097370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:27.102785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00829d-a6fa-4991-b3dd-2db0acc7ed4b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:27.103555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00829d-a6fa-4991-b3dd-2db0acc7ed4b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:27.103836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00829d-a6fa-4991-b3dd-2db0acc7ed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:27.179714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ed00829d-a6fa-4991-b3dd-2db0acc7ed4b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:27.180928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ed00829d-a6fa-4991-b3dd-2db0acc7ed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:27.207579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:27.281612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:27.430235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab2fbb-d053-4c52-9e5c-fc3480563463 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:27.431754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1aab2fbb-d053-4c52-9e5c-fc3480563463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:27.574678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab2fbb-d053-4c52-9e5c-fc3480563463 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:27.614285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:27.686176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bf3ad-41e8-477d-ad97-818e181f4fc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:27.686823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf3ad-41e8-477d-ad97-818e181f4fc6 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:27.687102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf3ad-41e8-477d-ad97-818e181f4fc6 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:27.687375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf3ad-41e8-477d-ad97-818e181f4fc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:27.689304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bb4bf3ad-41e8-477d-ad97-818e181f4fc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:27.776587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bf3ad-41e8-477d-ad97-818e181f4fc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:27.778187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bf3ad-41e8-477d-ad97-818e181f4fc6 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:27.795694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 700/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:28.679562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e73b1-eb31-49d7-8a48-eed9130941dd None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:28.680128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e73b1-eb31-49d7-8a48-eed9130941dd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:28.680128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e73b1-eb31-49d7-8a48-eed9130941dd None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:28.681529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:27:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:28.686404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133cd57-5952-4152-9eaa-a0b0c11303c3 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:28.686855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133cd57-5952-4152-9eaa-a0b0c11303c3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:28.686967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133cd57-5952-4152-9eaa-a0b0c11303c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:28.741447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:28.763290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0133cd57-5952-4152-9eaa-a0b0c11303c3 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:28.764525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0133cd57-5952-4152-9eaa-a0b0c11303c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:28.862629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:29.015714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e73b1-eb31-49d7-8a48-eed9130941dd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:29.017645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dc4e73b1-eb31-49d7-8a48-eed9130941dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:29.162643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e73b1-eb31-49d7-8a48-eed9130941dd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:29.174509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:29.275352 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ecfd1-b2dc-47be-8c2b-b85bc11f48d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:29.275996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ecfd1-b2dc-47be-8c2b-b85bc11f48d0 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:29.276273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ecfd1-b2dc-47be-8c2b-b85bc11f48d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:29.276578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ecfd1-b2dc-47be-8c2b-b85bc11f48d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:29.278492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0e1ecfd1-b2dc-47be-8c2b-b85bc11f48d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:29.365000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ecfd1-b2dc-47be-8c2b-b85bc11f48d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:29.366650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ecfd1-b2dc-47be-8c2b-b85bc11f48d0 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:29.384937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 721/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:30.249442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8427f-e7a2-48ac-84a6-d71161d619d9 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:30.249975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8427f-e7a2-48ac-84a6-d71161d619d9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:30.249975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8427f-e7a2-48ac-84a6-d71161d619d9 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:30.251310 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:30.256815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd8db0-0144-41d5-9585-f4f763d3eb20 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:30.257229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd8db0-0144-41d5-9585-f4f763d3eb20 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:30.257367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd8db0-0144-41d5-9585-f4f763d3eb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:30.276075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:30.332348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9bcd8db0-0144-41d5-9585-f4f763d3eb20 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:30.333573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9bcd8db0-0144-41d5-9585-f4f763d3eb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:30.430452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:30.578240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8427f-e7a2-48ac-84a6-d71161d619d9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:30.579712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-17d8427f-e7a2-48ac-84a6-d71161d619d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:30.687723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:30.723133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8427f-e7a2-48ac-84a6-d71161d619d9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:30.833991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f57ac-db79-4bd6-a3cb-cc450817fd03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:30.834570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f57ac-db79-4bd6-a3cb-cc450817fd03 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:30.834856 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f57ac-db79-4bd6-a3cb-cc450817fd03 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:30.835139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f57ac-db79-4bd6-a3cb-cc450817fd03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:30.836848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b21f57ac-db79-4bd6-a3cb-cc450817fd03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:30.910363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:30.924899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f57ac-db79-4bd6-a3cb-cc450817fd03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:30.926481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f57ac-db79-4bd6-a3cb-cc450817fd03 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:30.945577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 703/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:31.810996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b8643-6cec-4122-8cec-09d38528733f None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:31.811658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b8643-6cec-4122-8cec-09d38528733f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:31.811658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b8643-6cec-4122-8cec-09d38528733f None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:31.812937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:31.819426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6b625-a481-4608-b41a-582b4c385cae None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:31.819862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6b625-a481-4608-b41a-582b4c385cae None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:31.819986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6b625-a481-4608-b41a-582b4c385cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:31.893656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-77b6b625-a481-4608-b41a-582b4c385cae None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:31.894884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-77b6b625-a481-4608-b41a-582b4c385cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:31.992338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:32.144959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b8643-6cec-4122-8cec-09d38528733f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:32.146475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fa7b8643-6cec-4122-8cec-09d38528733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:32.229347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:32.290361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b8643-6cec-4122-8cec-09d38528733f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:32.400901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995363f-173d-4acb-bba8-def255e9edfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:32.401468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995363f-173d-4acb-bba8-def255e9edfb service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:32.401746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995363f-173d-4acb-bba8-def255e9edfb service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:32.402018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995363f-173d-4acb-bba8-def255e9edfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:32.403635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c995363f-173d-4acb-bba8-def255e9edfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:32.450031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:32.491069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995363f-173d-4acb-bba8-def255e9edfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:32.492694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995363f-173d-4acb-bba8-def255e9edfb service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:32.510837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 724/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:33.308428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4910b-fdcb-4682-8f62-123965eaf4c5 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:33.308967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4910b-fdcb-4682-8f62-123965eaf4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:33.308967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4910b-fdcb-4682-8f62-123965eaf4c5 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:33.310386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:33.315627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db96a-af7e-459b-9345-249cbaebe49f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:33.316020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db96a-af7e-459b-9345-249cbaebe49f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:33.316180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db96a-af7e-459b-9345-249cbaebe49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:33.389263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e60db96a-af7e-459b-9345-249cbaebe49f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:33.390462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e60db96a-af7e-459b-9345-249cbaebe49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:33.486202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:33.636757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4910b-fdcb-4682-8f62-123965eaf4c5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:33.638288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1ce4910b-fdcb-4682-8f62-123965eaf4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:33.687123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:33.784399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4910b-fdcb-4682-8f62-123965eaf4c5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:33.894091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0656b9-ceda-46c4-9fb2-ba937bd4035f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:33.894743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0656b9-ceda-46c4-9fb2-ba937bd4035f service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:33.895026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0656b9-ceda-46c4-9fb2-ba937bd4035f service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:33.895299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0656b9-ceda-46c4-9fb2-ba937bd4035f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:33.897575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bf0656b9-ceda-46c4-9fb2-ba937bd4035f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:33.904419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:33.989602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0656b9-ceda-46c4-9fb2-ba937bd4035f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:33.991272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0656b9-ceda-46c4-9fb2-ba937bd4035f service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:34.010489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 706/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:34.820226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c952341-323a-463b-af48-4328fc8de273 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:34.820841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c952341-323a-463b-af48-4328fc8de273 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:34.820841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c952341-323a-463b-af48-4328fc8de273 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:34.822203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:34.827395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1d75-4c8e-4a2f-998a-4ceeba48f11b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:34.827821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1d75-4c8e-4a2f-998a-4ceeba48f11b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:34.827988 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1d75-4c8e-4a2f-998a-4ceeba48f11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:34.903688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cb5f1d75-4c8e-4a2f-998a-4ceeba48f11b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:34.904926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cb5f1d75-4c8e-4a2f-998a-4ceeba48f11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:34.973176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:35.002480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:35.156822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c952341-323a-463b-af48-4328fc8de273 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:35.158396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7c952341-323a-463b-af48-4328fc8de273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:35.175700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:35.307107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c952341-323a-463b-af48-4328fc8de273 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:35.394376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:35.421035 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a99fe-0048-4a5c-8761-c5669bfa3de2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:35.421626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a99fe-0048-4a5c-8761-c5669bfa3de2 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:35.421908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a99fe-0048-4a5c-8761-c5669bfa3de2 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:35.422186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a99fe-0048-4a5c-8761-c5669bfa3de2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:35.423824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bf2a99fe-0048-4a5c-8761-c5669bfa3de2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:35.511045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a99fe-0048-4a5c-8761-c5669bfa3de2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:35.512677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a99fe-0048-4a5c-8761-c5669bfa3de2 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:35.530800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 727/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:36.334165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e12c2e-76e7-47cc-a337-21b63f5b6c1a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:36.334697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e12c2e-76e7-47cc-a337-21b63f5b6c1a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:36.334789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e12c2e-76e7-47cc-a337-21b63f5b6c1a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:36.336117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:27:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:36.341169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcecce-26c8-4d86-89a5-9be4d23e5329 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:36.341546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcecce-26c8-4d86-89a5-9be4d23e5329 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:36.341759 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcecce-26c8-4d86-89a5-9be4d23e5329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:36.416977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-fdbcecce-26c8-4d86-89a5-9be4d23e5329 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:36.418221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fdbcecce-26c8-4d86-89a5-9be4d23e5329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:36.459630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:36.514925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:36.670922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e12c2e-76e7-47cc-a337-21b63f5b6c1a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:36.672482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a4e12c2e-76e7-47cc-a337-21b63f5b6c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:36.813821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e12c2e-76e7-47cc-a337-21b63f5b6c1a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:36.866410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:36.925432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cde34-f1ae-4991-84ea-45e6ac7dbb2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:36.926095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cde34-f1ae-4991-84ea-45e6ac7dbb2b service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:36.926379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cde34-f1ae-4991-84ea-45e6ac7dbb2b service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:36.926656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cde34-f1ae-4991-84ea-45e6ac7dbb2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:36.928406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6f4cde34-f1ae-4991-84ea-45e6ac7dbb2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:37.017168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cde34-f1ae-4991-84ea-45e6ac7dbb2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:37.018769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cde34-f1ae-4991-84ea-45e6ac7dbb2b service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:37.037513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 709/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:37.844229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915143d-7b32-4481-9f2c-37c8ec7ff5b2 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:37.844841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915143d-7b32-4481-9f2c-37c8ec7ff5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:37.844841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915143d-7b32-4481-9f2c-37c8ec7ff5b2 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:37.846207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:37.851960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20877726-1d60-42da-8249-97e668539971 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:37.852387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20877726-1d60-42da-8249-97e668539971 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:37.852535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20877726-1d60-42da-8249-97e668539971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:37.927493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-20877726-1d60-42da-8249-97e668539971 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:37.928878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-20877726-1d60-42da-8249-97e668539971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:37.930925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:38.024425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:38.175703 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915143d-7b32-4481-9f2c-37c8ec7ff5b2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:38.177314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a915143d-7b32-4481-9f2c-37c8ec7ff5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:38.320854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915143d-7b32-4481-9f2c-37c8ec7ff5b2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:38.347406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:38.432730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b4f5e-9d69-43c4-9b9e-422f55cb05ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:38.433322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b4f5e-9d69-43c4-9b9e-422f55cb05ad service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:38.433608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b4f5e-9d69-43c4-9b9e-422f55cb05ad service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:38.433884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b4f5e-9d69-43c4-9b9e-422f55cb05ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:38.435514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d68b4f5e-9d69-43c4-9b9e-422f55cb05ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:38.523575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b4f5e-9d69-43c4-9b9e-422f55cb05ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:38.525205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b4f5e-9d69-43c4-9b9e-422f55cb05ad service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:38.542796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 730/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:40.486070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e3a6e-570b-4821-b116-13df8b451264 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:40.486642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e3a6e-570b-4821-b116-13df8b451264 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:40.486642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e3a6e-570b-4821-b116-13df8b451264 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:40.487989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:40.493836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36363ce3-5e62-499c-9fee-8b60aaf06fe4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:40.494258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36363ce3-5e62-499c-9fee-8b60aaf06fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:40.494400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36363ce3-5e62-499c-9fee-8b60aaf06fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:40.533522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:40.559143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-36363ce3-5e62-499c-9fee-8b60aaf06fe4 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:40.560365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-36363ce3-5e62-499c-9fee-8b60aaf06fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:40.656846 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:27:40.696591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e3a6e-570b-4821-b116-13df8b451264 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:40.698132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b62e3a6e-570b-4821-b116-13df8b451264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:40.801118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e3a6e-570b-4821-b116-13df8b451264 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:40.833103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:40.874855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9297ea-543d-4869-92cc-7ea59aef14de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:40.875508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9297ea-543d-4869-92cc-7ea59aef14de service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:40.875790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9297ea-543d-4869-92cc-7ea59aef14de service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:40.876065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9297ea-543d-4869-92cc-7ea59aef14de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:40.877783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cb9297ea-543d-4869-92cc-7ea59aef14de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:40.986547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9297ea-543d-4869-92cc-7ea59aef14de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:40.987970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9297ea-543d-4869-92cc-7ea59aef14de service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:41.006023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 712/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:43.207165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea148675-98a1-4cd1-9631-007e911ed341 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:43.207698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea148675-98a1-4cd1-9631-007e911ed341 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:43.207698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea148675-98a1-4cd1-9631-007e911ed341 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:43.209033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:43.214355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bde2b-5746-4012-baa0-b89ba5823033 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:43.214738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bde2b-5746-4012-baa0-b89ba5823033 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:43.214917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bde2b-5746-4012-baa0-b89ba5823033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:43.279239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-793bde2b-5746-4012-baa0-b89ba5823033 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:43.280489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-793bde2b-5746-4012-baa0-b89ba5823033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:43.293051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:43.378751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:43.417921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea148675-98a1-4cd1-9631-007e911ed341 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:43.419432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ea148675-98a1-4cd1-9631-007e911ed341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:43.523733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea148675-98a1-4cd1-9631-007e911ed341 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:43.535600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:43.603129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050fdc8-e6c2-486b-a33c-cece8e731098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:43.603776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050fdc8-e6c2-486b-a33c-cece8e731098 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:43.604059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050fdc8-e6c2-486b-a33c-cece8e731098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:43.604364 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050fdc8-e6c2-486b-a33c-cece8e731098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:43.606000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e050fdc8-e6c2-486b-a33c-cece8e731098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:43.715383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050fdc8-e6c2-486b-a33c-cece8e731098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:43.716853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050fdc8-e6c2-486b-a33c-cece8e731098 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:43.735628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 733/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:46.227242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019012cd-f6b3-43cc-b0fc-9fbe257f4ebd None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:46.227772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019012cd-f6b3-43cc-b0fc-9fbe257f4ebd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:46.227772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019012cd-f6b3-43cc-b0fc-9fbe257f4ebd None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:46.229125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:46.234411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bfc93-9cd7-4a7b-b0ce-81e1afc087ca None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:46.234825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bfc93-9cd7-4a7b-b0ce-81e1afc087ca None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:46.234966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bfc93-9cd7-4a7b-b0ce-81e1afc087ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:46.281281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:46.298722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4b0bfc93-9cd7-4a7b-b0ce-81e1afc087ca None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:46.299914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4b0bfc93-9cd7-4a7b-b0ce-81e1afc087ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:46.395605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:46.435087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019012cd-f6b3-43cc-b0fc-9fbe257f4ebd None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:46.436710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-019012cd-f6b3-43cc-b0fc-9fbe257f4ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:46.540442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019012cd-f6b3-43cc-b0fc-9fbe257f4ebd None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:46.580444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:46.614676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f15f9-67b9-415a-b830-420bacc586e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:46.615943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f15f9-67b9-415a-b830-420bacc586e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:46.616259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f15f9-67b9-415a-b830-420bacc586e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:46.616567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f15f9-67b9-415a-b830-420bacc586e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:46.618227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1d9f15f9-67b9-415a-b830-420bacc586e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:46.728587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f15f9-67b9-415a-b830-420bacc586e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:46.730023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f15f9-67b9-415a-b830-420bacc586e5 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:46.747979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 715/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:48.930794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d708978-7ba9-48f5-9868-b0f809da174d None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:48.931326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d708978-7ba9-48f5-9868-b0f809da174d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:48.931326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d708978-7ba9-48f5-9868-b0f809da174d None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:48.932771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:48.938214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26d258-a0a1-4ce0-9f3c-c95191469392 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:48.938661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26d258-a0a1-4ce0-9f3c-c95191469392 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:48.938758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26d258-a0a1-4ce0-9f3c-c95191469392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:49.005124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2f26d258-a0a1-4ce0-9f3c-c95191469392 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:49.006832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2f26d258-a0a1-4ce0-9f3c-c95191469392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:49.027746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:49.104356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:49.142887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d708978-7ba9-48f5-9868-b0f809da174d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:49.144446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5d708978-7ba9-48f5-9868-b0f809da174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:49.247054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d708978-7ba9-48f5-9868-b0f809da174d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:49.266142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:49.320735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9413e7a-7432-4f97-8b55-0eed89073272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:49.321372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9413e7a-7432-4f97-8b55-0eed89073272 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:49.321654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9413e7a-7432-4f97-8b55-0eed89073272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:49.321928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9413e7a-7432-4f97-8b55-0eed89073272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:49.323553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e9413e7a-7432-4f97-8b55-0eed89073272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:49.430621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9413e7a-7432-4f97-8b55-0eed89073272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:49.432064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9413e7a-7432-4f97-8b55-0eed89073272 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:49.450709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 736/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:51.659417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2188b5-1231-4835-8ce6-186d0b8e3fca None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:51.659993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2188b5-1231-4835-8ce6-186d0b8e3fca None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:51.659993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2188b5-1231-4835-8ce6-186d0b8e3fca None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:51.661418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:51.666808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41d9d-70ff-41ae-ac64-604366f44195 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:51.667241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41d9d-70ff-41ae-ac64-604366f44195 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:51.667364 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41d9d-70ff-41ae-ac64-604366f44195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:51.728830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:51.731608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a5e41d9d-70ff-41ae-ac64-604366f44195 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:51.733474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a5e41d9d-70ff-41ae-ac64-604366f44195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:51.828741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:51.867065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2188b5-1231-4835-8ce6-186d0b8e3fca None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:51.868743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-dd2188b5-1231-4835-8ce6-186d0b8e3fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:51.973874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2188b5-1231-4835-8ce6-186d0b8e3fca None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:52.022705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:52.048496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5f22c-4cb5-4af5-aa48-ed0ed78e5bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:52.049153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5f22c-4cb5-4af5-aa48-ed0ed78e5bad service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:52.049439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5f22c-4cb5-4af5-aa48-ed0ed78e5bad service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:52.049716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5f22c-4cb5-4af5-aa48-ed0ed78e5bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:52.051384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-23e5f22c-4cb5-4af5-aa48-ed0ed78e5bad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:52.158271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5f22c-4cb5-4af5-aa48-ed0ed78e5bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:52.159694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5f22c-4cb5-4af5-aa48-ed0ed78e5bad service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:52.177106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 718/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:54.307031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13173d4f-3473-4b70-af89-e7a89742dc9d None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:54.307539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13173d4f-3473-4b70-af89-e7a89742dc9d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:54.307539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13173d4f-3473-4b70-af89-e7a89742dc9d None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:54.308893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:54.314487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9134c-5bea-4094-918c-0de265894552 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:54.314933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9134c-5bea-4094-918c-0de265894552 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:54.315040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9134c-5bea-4094-918c-0de265894552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:54.381424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-92b9134c-5bea-4094-918c-0de265894552 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:54.382661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-92b9134c-5bea-4094-918c-0de265894552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:54.411137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:54.479067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:54.516518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13173d4f-3473-4b70-af89-e7a89742dc9d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:54.518046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-13173d4f-3473-4b70-af89-e7a89742dc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:54.620096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13173d4f-3473-4b70-af89-e7a89742dc9d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:54.649709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:54.694424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bce38-7855-4178-864f-3a60d7c26b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:54.695082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bce38-7855-4178-864f-3a60d7c26b2f service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:54.695367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bce38-7855-4178-864f-3a60d7c26b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:54.695645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bce38-7855-4178-864f-3a60d7c26b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:54.697347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-612bce38-7855-4178-864f-3a60d7c26b2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:54.804192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bce38-7855-4178-864f-3a60d7c26b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:54.805641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bce38-7855-4178-864f-3a60d7c26b2f service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:54.824047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 739/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:56.922134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552235c-d51b-4535-89f8-c27019904fbd None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:56.922625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552235c-d51b-4535-89f8-c27019904fbd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:56.922750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552235c-d51b-4535-89f8-c27019904fbd None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:56.924073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:56.930164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f4b8e-f391-468f-a355-eea88cf3844e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:56.930570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f4b8e-f391-468f-a355-eea88cf3844e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:56.930744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f4b8e-f391-468f-a355-eea88cf3844e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:56.995447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-af6f4b8e-f391-468f-a355-eea88cf3844e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:56.996672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-af6f4b8e-f391-468f-a355-eea88cf3844e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:57.004740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:57.092062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:57.130394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552235c-d51b-4535-89f8-c27019904fbd None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:57.131961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c552235c-d51b-4535-89f8-c27019904fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:57.242503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552235c-d51b-4535-89f8-c27019904fbd None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:57.299157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:57.317781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836119b-508e-413e-845a-5b82515cb620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:57.318430 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836119b-508e-413e-845a-5b82515cb620 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:57.318714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836119b-508e-413e-845a-5b82515cb620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:57.319558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836119b-508e-413e-845a-5b82515cb620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:57.321270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0836119b-508e-413e-845a-5b82515cb620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:57.429645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836119b-508e-413e-845a-5b82515cb620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:27:57.431064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836119b-508e-413e-845a-5b82515cb620 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:27:57.448802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 721/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:27:59.539106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9ba69-3320-4263-bc27-04aeca31c57f None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:59.539634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9ba69-3320-4263-bc27-04aeca31c57f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:59.539634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9ba69-3320-4263-bc27-04aeca31c57f None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:59.540994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:27:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:27:59.546275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79328d18-a1c2-4719-aa8d-e78896b7e282 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:59.546708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79328d18-a1c2-4719-aa8d-e78896b7e282 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:59.546834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79328d18-a1c2-4719-aa8d-e78896b7e282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:59.612880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-79328d18-a1c2-4719-aa8d-e78896b7e282 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:27:59.614083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-79328d18-a1c2-4719-aa8d-e78896b7e282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:59.652871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:59.711433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:27:59.749264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9ba69-3320-4263-bc27-04aeca31c57f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:27:59.750781 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9ca9ba69-3320-4263-bc27-04aeca31c57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:27:59.852998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9ba69-3320-4263-bc27-04aeca31c57f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:27:59.889340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:27:59.925418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e361ae0-11b0-47c2-8629-ae37d4060d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:27:59.926065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361ae0-11b0-47c2-8629-ae37d4060d22 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:27:59.926353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361ae0-11b0-47c2-8629-ae37d4060d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:27:59.926628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361ae0-11b0-47c2-8629-ae37d4060d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:27:59.928281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0e361ae0-11b0-47c2-8629-ae37d4060d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:00.038507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e361ae0-11b0-47c2-8629-ae37d4060d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:28:00.039960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e361ae0-11b0-47c2-8629-ae37d4060d22 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:28:00.058256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 742/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:28:02.114875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd306cd-58f9-48e5-8832-005b27f6f24a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:02.115441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd306cd-58f9-48e5-8832-005b27f6f24a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:02.115441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd306cd-58f9-48e5-8832-005b27f6f24a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:02.117332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:28:02.122515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769a818-00e5-43dd-a895-0a958a528e28 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:02.122920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769a818-00e5-43dd-a895-0a958a528e28 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:02.123078 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769a818-00e5-43dd-a895-0a958a528e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:02.192017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d769a818-00e5-43dd-a895-0a958a528e28 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:28:02.193175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d769a818-00e5-43dd-a895-0a958a528e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:02.210638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:02.291586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:28:02.330267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd306cd-58f9-48e5-8832-005b27f6f24a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:28:02.331811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1bd306cd-58f9-48e5-8832-005b27f6f24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:02.435257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd306cd-58f9-48e5-8832-005b27f6f24a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:28:02.501630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:02.509042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fae63f-75ef-4921-890c-3d1452445b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:28:02.509756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fae63f-75ef-4921-890c-3d1452445b08 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:02.510064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fae63f-75ef-4921-890c-3d1452445b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:02.510378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fae63f-75ef-4921-890c-3d1452445b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:02.512067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-07fae63f-75ef-4921-890c-3d1452445b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:02.622575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fae63f-75ef-4921-890c-3d1452445b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:28:02.624055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fae63f-75ef-4921-890c-3d1452445b08 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:28:02.642279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 724/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:28:04.690454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29279cb-ea25-40ec-9ec4-4262a23ef179 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:04.691022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29279cb-ea25-40ec-9ec4-4262a23ef179 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:04.691022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29279cb-ea25-40ec-9ec4-4262a23ef179 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:04.692669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:28:04.698592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75a740-c7a6-49ca-b7f3-58dc550c69bc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:04.699001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75a740-c7a6-49ca-b7f3-58dc550c69bc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:04.699147 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75a740-c7a6-49ca-b7f3-58dc550c69bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:04.762055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2b75a740-c7a6-49ca-b7f3-58dc550c69bc None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:28:04.763244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2b75a740-c7a6-49ca-b7f3-58dc550c69bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:04.810165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:04.859411 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:28:04.895520 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29279cb-ea25-40ec-9ec4-4262a23ef179 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:28:04.897838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b29279cb-ea25-40ec-9ec4-4262a23ef179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:05.002130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29279cb-ea25-40ec-9ec4-4262a23ef179 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:28:05.048393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:05.073716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b0ef7-6224-41a5-be4f-efcdbfab8069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:28:05.074354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b0ef7-6224-41a5-be4f-efcdbfab8069 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:05.074634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b0ef7-6224-41a5-be4f-efcdbfab8069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:05.074905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b0ef7-6224-41a5-be4f-efcdbfab8069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:05.076563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8d2b0ef7-6224-41a5-be4f-efcdbfab8069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:05.184368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b0ef7-6224-41a5-be4f-efcdbfab8069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:28:05.185800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b0ef7-6224-41a5-be4f-efcdbfab8069 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:28:05.204060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 745/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:28:14.857649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=22 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:15.201912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=66 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:25.011111 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f45f2a-aed9-4819-a2c4-d8219829044d None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:25.011632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f45f2a-aed9-4819-a2c4-d8219829044d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:25.011749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f45f2a-aed9-4819-a2c4-d8219829044d None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:25.013086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 726/1471] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:28:25.018365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf29e00-fc12-4075-a257-f6c88f96cb96 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:25.018763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf29e00-fc12-4075-a257-f6c88f96cb96 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:25.018913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf29e00-fc12-4075-a257-f6c88f96cb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:25.083572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9cf29e00-fc12-4075-a257-f6c88f96cb96 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:28:25.084791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9cf29e00-fc12-4075-a257-f6c88f96cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:25.182078 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 746/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:28:25.265900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14af671-1c4e-45f5-bcd6-3e5c72546f06 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:25.266279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14af671-1c4e-45f5-bcd6-3e5c72546f06 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:25.266600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14af671-1c4e-45f5-bcd6-3e5c72546f06 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:25.268039 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 727/1473] 10.222.0.57 () {54 vars in 966 bytes} [Thu Jul 21 17:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:28:25.272913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d211a70-65a7-462f-a256-997d26a909f7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:25.273200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d211a70-65a7-462f-a256-997d26a909f7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:25.273476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d211a70-65a7-462f-a256-997d26a909f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:25.347615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8d211a70-65a7-462f-a256-997d26a909f7 None None] MFA Rules not processed for user `9e07c59babad479cadbf74f15ee5eea9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:28:25.348889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8d211a70-65a7-462f-a256-997d26a909f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:25.369955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:25.425332 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 747/1474] 10.222.0.57 () {58 vars in 1063 bytes} [Thu Jul 21 17:28:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:28:25.433424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14af671-1c4e-45f5-bcd6-3e5c72546f06 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:28:25.434967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a14af671-1c4e-45f5-bcd6-3e5c72546f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:25.542254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14af671-1c4e-45f5-bcd6-3e5c72546f06 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:28:25.615426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfccff-588a-4963-a094-b2c992a4aa9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e07c59babad479cadbf74f15ee5eea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:28:25.616075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfccff-588a-4963-a094-b2c992a4aa9b service glance] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:25.616955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfccff-588a-4963-a094-b2c992a4aa9b service glance] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:25.617243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfccff-588a-4963-a094-b2c992a4aa9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:25.618910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-92cfccff-588a-4963-a094-b2c992a4aa9b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:25.672206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:25.729479 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfccff-588a-4963-a094-b2c992a4aa9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:28:25.730931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfccff-588a-4963-a094-b2c992a4aa9b service glance] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:28:25.750519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 728/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:28:27.867006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1604b8-7f13-4a9c-8516-579e3e6ec73d None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:27.867553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1604b8-7f13-4a9c-8516-579e3e6ec73d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:27.867553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1604b8-7f13-4a9c-8516-579e3e6ec73d None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:27.868950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 748/1476] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:28:27.874144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77865b-6e9b-4c44-8037-38a9f7876d38 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:27.874571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77865b-6e9b-4c44-8037-38a9f7876d38 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:27.874733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77865b-6e9b-4c44-8037-38a9f7876d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:27.940087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7f77865b-6e9b-4c44-8037-38a9f7876d38 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:28:27.941347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7f77865b-6e9b-4c44-8037-38a9f7876d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:28.038329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 729/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:28:28.061461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1604b8-7f13-4a9c-8516-579e3e6ec73d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:28:28.062996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9a1604b8-7f13-4a9c-8516-579e3e6ec73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:28.185924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1604b8-7f13-4a9c-8516-579e3e6ec73d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:28:28.235370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:28.280222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298410db-1f98-4115-95b6-53a3c6ae0e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:28:28.281137 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298410db-1f98-4115-95b6-53a3c6ae0e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:28.281421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298410db-1f98-4115-95b6-53a3c6ae0e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:28.281697 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298410db-1f98-4115-95b6-53a3c6ae0e3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:28.282433 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298410db-1f98-4115-95b6-53a3c6ae0e3f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:28:28.284368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-298410db-1f98-4115-95b6-53a3c6ae0e3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:28:28.284368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 21 17:28:28.284368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: Jul 21 17:28:28.285541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298410db-1f98-4115-95b6-53a3c6ae0e3f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:28:28.305679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 749/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Jul 21 17:28:28 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:28:30.364893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ce67-25a3-4d70-9d9b-2609806de8f1 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:30.365447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ce67-25a3-4d70-9d9b-2609806de8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:30.365447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ce67-25a3-4d70-9d9b-2609806de8f1 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:30.367362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 730/1479] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:28:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:28:30.372782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9034c-cac0-46a4-89de-39cb95c895e8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:30.373211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9034c-cac0-46a4-89de-39cb95c895e8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:30.373364 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9034c-cac0-46a4-89de-39cb95c895e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:30.437510 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d9f9034c-cac0-46a4-89de-39cb95c895e8 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:28:30.438705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d9f9034c-cac0-46a4-89de-39cb95c895e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:30.535908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 750/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:28:30.579774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100ce67-25a3-4d70-9d9b-2609806de8f1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:28:30.581533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9100ce67-25a3-4d70-9d9b-2609806de8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:30.591366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:30.730285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100ce67-25a3-4d70-9d9b-2609806de8f1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:28:30.809076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:30.841850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c2392-fcf0-46ca-8769-b6a9cf0710d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:28:30.842506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c2392-fcf0-46ca-8769-b6a9cf0710d4 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:30.842791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c2392-fcf0-46ca-8769-b6a9cf0710d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:30.843071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c2392-fcf0-46ca-8769-b6a9cf0710d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:30.844751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-105c2392-fcf0-46ca-8769-b6a9cf0710d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:30.933460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c2392-fcf0-46ca-8769-b6a9cf0710d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:28:30.935069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c2392-fcf0-46ca-8769-b6a9cf0710d4 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:28:30.953214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 731/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:28:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:28:33.156599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfeee6-f989-4765-831b-dbea7c5848e7 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:33.157179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfeee6-f989-4765-831b-dbea7c5848e7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:33.157179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfeee6-f989-4765-831b-dbea7c5848e7 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:33.158509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 751/1482] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:28:33.163701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1e454-fd5b-404a-afb1-07121efd0688 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:33.164098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1e454-fd5b-404a-afb1-07121efd0688 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:33.164829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1e454-fd5b-404a-afb1-07121efd0688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:33.230151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5ee1e454-fd5b-404a-afb1-07121efd0688 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:28:33.231364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5ee1e454-fd5b-404a-afb1-07121efd0688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:33.277445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:33.327429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 732/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:28:33.364239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfeee6-f989-4765-831b-dbea7c5848e7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:28:33.365776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-08bfeee6-f989-4765-831b-dbea7c5848e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:33.388656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:33.470488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfeee6-f989-4765-831b-dbea7c5848e7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:28:33.549921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6bc8a-87b1-4a68-a303-767efd77216c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:28:33.550579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6bc8a-87b1-4a68-a303-767efd77216c service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:33.550865 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6bc8a-87b1-4a68-a303-767efd77216c service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:33.551142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6bc8a-87b1-4a68-a303-767efd77216c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:33.552863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5fd6bc8a-87b1-4a68-a303-767efd77216c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:33.626985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:33.663791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6bc8a-87b1-4a68-a303-767efd77216c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:28:33.665436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6bc8a-87b1-4a68-a303-767efd77216c service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:28:33.683719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 752/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:28:35.755817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea55d-3612-4cff-a322-47ffcac2101f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:35.756404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea55d-3612-4cff-a322-47ffcac2101f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:35.756404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea55d-3612-4cff-a322-47ffcac2101f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:35.757880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 733/1485] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:28:35.763675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e252e49-dea2-4efd-8736-3de2f65b9141 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:35.764113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e252e49-dea2-4efd-8736-3de2f65b9141 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:35.764221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e252e49-dea2-4efd-8736-3de2f65b9141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:35.830060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0e252e49-dea2-4efd-8736-3de2f65b9141 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:28:35.831272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0e252e49-dea2-4efd-8736-3de2f65b9141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:35.925243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 753/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:28:35.964549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ea55d-3612-4cff-a322-47ffcac2101f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:28:35.966112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b05ea55d-3612-4cff-a322-47ffcac2101f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:36.068040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ea55d-3612-4cff-a322-47ffcac2101f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:28:36.140230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:36.142264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa967d5-24f5-4c90-b217-d177223373e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:28:36.142950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa967d5-24f5-4c90-b217-d177223373e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:36.143513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa967d5-24f5-4c90-b217-d177223373e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:36.144065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa967d5-24f5-4c90-b217-d177223373e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:36.145928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3fa967d5-24f5-4c90-b217-d177223373e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:36.256438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa967d5-24f5-4c90-b217-d177223373e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:28:36.257869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa967d5-24f5-4c90-b217-d177223373e7 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:28:36.276383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 734/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:28:38.461111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a264-3446-4f79-a578-dd25aeedab0e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:38.462156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a264-3446-4f79-a578-dd25aeedab0e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:38.462288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a264-3446-4f79-a578-dd25aeedab0e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:38.463657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 754/1488] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:28:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:28:38.468878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000403f-f4fc-4e13-9004-7b1646d43dec None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:38.469322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000403f-f4fc-4e13-9004-7b1646d43dec None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:38.469508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000403f-f4fc-4e13-9004-7b1646d43dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:38.534067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4000403f-f4fc-4e13-9004-7b1646d43dec None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:28:38.535265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4000403f-f4fc-4e13-9004-7b1646d43dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:38.590953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:38.752746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 735/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:28:38.793735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a264-3446-4f79-a578-dd25aeedab0e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:28:38.795282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a025a264-3446-4f79-a578-dd25aeedab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:38.835129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:38.902809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a264-3446-4f79-a578-dd25aeedab0e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:28:38.978016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e78970-393a-4398-b468-295cfbc75bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:28:38.978663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e78970-393a-4398-b468-295cfbc75bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:38.978970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e78970-393a-4398-b468-295cfbc75bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:38.979244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e78970-393a-4398-b468-295cfbc75bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:38.980942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-87e78970-393a-4398-b468-295cfbc75bc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:39.059976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:39.090002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e78970-393a-4398-b468-295cfbc75bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:28:39.091439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e78970-393a-4398-b468-295cfbc75bc9 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:28:39.109059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 755/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:28:41.441120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83f4fa-4e87-46b7-99aa-e647eddc05b4 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:41.441955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83f4fa-4e87-46b7-99aa-e647eddc05b4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:41.442149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83f4fa-4e87-46b7-99aa-e647eddc05b4 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:41.443909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 736/1491] 10.222.0.57 () {54 vars in 926 bytes} [Thu Jul 21 17:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:28:41.449338 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab00311-5b12-412e-a25e-3eb307cfbfbd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:41.449762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab00311-5b12-412e-a25e-3eb307cfbfbd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:41.449952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab00311-5b12-412e-a25e-3eb307cfbfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:41.519512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0ab00311-5b12-412e-a25e-3eb307cfbfbd None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:28:41.520723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0ab00311-5b12-412e-a25e-3eb307cfbfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:41.622895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 756/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Thu Jul 21 17:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:28:41.662678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83f4fa-4e87-46b7-99aa-e647eddc05b4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:28:41.664247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7e83f4fa-4e87-46b7-99aa-e647eddc05b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:41.766227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83f4fa-4e87-46b7-99aa-e647eddc05b4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:28:41.839784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853ae17-f459-4a7d-aae7-f61f75eb4a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:28:41.840377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ae17-f459-4a7d-aae7-f61f75eb4a94 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:28:41.840659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ae17-f459-4a7d-aae7-f61f75eb4a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:28:41.840931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ae17-f459-4a7d-aae7-f61f75eb4a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:28:41.842609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7853ae17-f459-4a7d-aae7-f61f75eb4a94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:28:41.851091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:41.953843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853ae17-f459-4a7d-aae7-f61f75eb4a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:28:41.955284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853ae17-f459-4a7d-aae7-f61f75eb4a94 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:28:41.973651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 737/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:28:51.621055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=82 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:28:51.971303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=50 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:11.615270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6617-c91c-4a1c-90f9-3d0a96d981a1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:11.615816 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6617-c91c-4a1c-90f9-3d0a96d981a1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:11.615816 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6617-c91c-4a1c-90f9-3d0a96d981a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:11.691261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d40b6617-c91c-4a1c-90f9-3d0a96d981a1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:30:11.692493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d40b6617-c91c-4a1c-90f9-3d0a96d981a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:11.788533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 757/1494] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:30:11.796826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573ba97-f5b0-4eec-abe4-f580b0dd5026 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:11.797332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573ba97-f5b0-4eec-abe4-f580b0dd5026 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:11.797437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573ba97-f5b0-4eec-abe4-f580b0dd5026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:11.871944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f573ba97-f5b0-4eec-abe4-f580b0dd5026 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:30:11.873185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f573ba97-f5b0-4eec-abe4-f580b0dd5026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:11.969412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 738/1495] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:30:11.974022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b6617-c91c-4a1c-90f9-3d0a96d981a1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:30:11.975208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d40b6617-c91c-4a1c-90f9-3d0a96d981a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:11.987067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b6617-c91c-4a1c-90f9-3d0a96d981a1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:30:12.047399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:12.081286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:30:12.081942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:12.082221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:12.082495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:12.083208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:30:12.083514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:30:12.084625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:30:12.084625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 21 17:30:12.084625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: Jul 21 17:30:12.085503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:30:12.096593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 758/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:30:12.105438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573ba97-f5b0-4eec-abe4-f580b0dd5026 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:30:12.114268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573ba97-f5b0-4eec-abe4-f580b0dd5026 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:30:12.173925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:12.209123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0ad1b-420c-419d-b42f-cb6a08aabded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:30:12.209808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0ad1b-420c-419d-b42f-cb6a08aabded admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:12.210090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0ad1b-420c-419d-b42f-cb6a08aabded admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:12.210363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0ad1b-420c-419d-b42f-cb6a08aabded admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:12.211056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0ad1b-420c-419d-b42f-cb6a08aabded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:30:12.212967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-80f0ad1b-420c-419d-b42f-cb6a08aabded admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:30:12.212967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 21 17:30:12.212967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: Jul 21 17:30:12.214153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0ad1b-420c-419d-b42f-cb6a08aabded admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:30:12.268468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 739/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:30:12 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:30:12.273064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:30:12.275227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:12.310043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff256be6-bb3b-4296-a3bd-2bc52ce90445 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:30:12.407260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a06081-b051-4358-9953-b7d3004b76d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:30:12.407927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a06081-b051-4358-9953-b7d3004b76d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:12.408207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a06081-b051-4358-9953-b7d3004b76d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:12.408528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a06081-b051-4358-9953-b7d3004b76d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:12.409221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a06081-b051-4358-9953-b7d3004b76d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:30:12.410232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-82a06081-b051-4358-9953-b7d3004b76d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:30:12.410232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 21 17:30:12.410232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: Jul 21 17:30:12.411439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a06081-b051-4358-9953-b7d3004b76d0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:30:12.451476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 759/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:30:12 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:30:12.455569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0ad1b-420c-419d-b42f-cb6a08aabded admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:30:12.457136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-80f0ad1b-420c-419d-b42f-cb6a08aabded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:12.519053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0ad1b-420c-419d-b42f-cb6a08aabded admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:30:12.604097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:12.615380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0edb6-e9a9-48fc-aa2d-f344bca0bb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:30:12.616047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0edb6-e9a9-48fc-aa2d-f344bca0bb62 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:12.616352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0edb6-e9a9-48fc-aa2d-f344bca0bb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:12.616626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0edb6-e9a9-48fc-aa2d-f344bca0bb62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:12.617344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0edb6-e9a9-48fc-aa2d-f344bca0bb62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:30:12.618494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-4ee0edb6-e9a9-48fc-aa2d-f344bca0bb62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:30:12.618494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 21 17:30:12.618494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: Jul 21 17:30:12.619373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0edb6-e9a9-48fc-aa2d-f344bca0bb62 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:30:12.631507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 740/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:30:12.636211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a06081-b051-4358-9953-b7d3004b76d0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:30:12.637730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-82a06081-b051-4358-9953-b7d3004b76d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:12.648904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a06081-b051-4358-9953-b7d3004b76d0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:30:12.673839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:12.745441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0894-73fc-4969-bd16-24781240e34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:30:12.746034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0894-73fc-4969-bd16-24781240e34b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:12.746315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0894-73fc-4969-bd16-24781240e34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:12.746590 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0894-73fc-4969-bd16-24781240e34b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:12.747329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f0894-73fc-4969-bd16-24781240e34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:30:12.748365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-0d6f0894-73fc-4969-bd16-24781240e34b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:30:12.748365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 21 17:30:12.748365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: Jul 21 17:30:12.749248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f0894-73fc-4969-bd16-24781240e34b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:30:12.762069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 760/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:30:12.766287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0edb6-e9a9-48fc-aa2d-f344bca0bb62 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:30:12.767870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4ee0edb6-e9a9-48fc-aa2d-f344bca0bb62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:12.801007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0edb6-e9a9-48fc-aa2d-f344bca0bb62 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:30:12.893121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad25496-9cff-49b0-905a-96cbf452ba0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:30:12.893998 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad25496-9cff-49b0-905a-96cbf452ba0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:12.894274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad25496-9cff-49b0-905a-96cbf452ba0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:12.894547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad25496-9cff-49b0-905a-96cbf452ba0b admin admin] PATH_INFO: `/v3/projects/d1ce5eb4989d46ee8738500709cf2dca/users/5a34690403de409ea5bf63b1918f4552/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:12.917981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad25496-9cff-49b0-905a-96cbf452ba0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ce5eb4989d46ee8738500709cf2dca, user_id=5a34690403de409ea5bf63b1918f4552, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:30:12.919003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-1ad25496-9cff-49b0-905a-96cbf452ba0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:30:12.919003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 21 17:30:12.919003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: Jul 21 17:30:12.920992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad25496-9cff-49b0-905a-96cbf452ba0b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:30:12.950751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 741/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:30:12 2022] PUT /identity/v3/projects/d1ce5eb4989d46ee8738500709cf2dca/users/5a34690403de409ea5bf63b1918f4552/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:30:12.963211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51174da-f030-48ba-9e9a-649d22b02780 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:12.963492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51174da-f030-48ba-9e9a-649d22b02780 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:12.963769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51174da-f030-48ba-9e9a-649d22b02780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:13.019363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b51174da-f030-48ba-9e9a-649d22b02780 None None] MFA Rules not processed for user `5a34690403de409ea5bf63b1918f4552`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:30:13.020528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b51174da-f030-48ba-9e9a-649d22b02780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:13.099484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 761/1502] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:30:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:30:13.112976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bcfd3-3e28-4c03-b222-0df0c55b72d6 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:13.113263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bcfd3-3e28-4c03-b222-0df0c55b72d6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:13.113538 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bcfd3-3e28-4c03-b222-0df0c55b72d6 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:13.114930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 742/1503] 10.222.0.57 () {54 vars in 964 bytes} [Thu Jul 21 17:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:30:13.120640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3277a-b25f-480e-8f06-9c2687d0f3f5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:13.120935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3277a-b25f-480e-8f06-9c2687d0f3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:13.121222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3277a-b25f-480e-8f06-9c2687d0f3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:13.151322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:13.195365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a6f3277a-b25f-480e-8f06-9c2687d0f3f5 None None] MFA Rules not processed for user `9bbddf0ea88049aa9bb6a257e1f01ffd`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:30:13.197432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a6f3277a-b25f-480e-8f06-9c2687d0f3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:13.272570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 762/1504] 10.222.0.57 () {58 vars in 1061 bytes} [Thu Jul 21 17:30:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:30:13.280646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bcfd3-3e28-4c03-b222-0df0c55b72d6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:30:13.282279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d73bcfd3-3e28-4c03-b222-0df0c55b72d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:13.336171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:13.392427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bcfd3-3e28-4c03-b222-0df0c55b72d6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:30:13.467313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93340fb9-1fdd-4b84-8735-9172dfb2d332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:30:13.467963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340fb9-1fdd-4b84-8735-9172dfb2d332 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:13.468252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340fb9-1fdd-4b84-8735-9172dfb2d332 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:13.468641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340fb9-1fdd-4b84-8735-9172dfb2d332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:13.470225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-93340fb9-1fdd-4b84-8735-9172dfb2d332 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:13.590469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93340fb9-1fdd-4b84-8735-9172dfb2d332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:30:13.592959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93340fb9-1fdd-4b84-8735-9172dfb2d332 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:30:13.596489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:13.612493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 743/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:30:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:30:13.656361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170cc39-dc27-40a1-9d29-c66cd8fa53a3 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:13.656817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170cc39-dc27-40a1-9d29-c66cd8fa53a3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:13.656979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170cc39-dc27-40a1-9d29-c66cd8fa53a3 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:13.658547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 763/1506] 10.222.0.57 () {52 vars in 829 bytes} [Thu Jul 21 17:30:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:30:13.661824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93340fb9-1fdd-4b84-8735-9172dfb2d332 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:30:13.663037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-93340fb9-1fdd-4b84-8735-9172dfb2d332 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:13.764404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93340fb9-1fdd-4b84-8735-9172dfb2d332 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:30:13.858975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:30:13.859762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:13.860041 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:13.860331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] PATH_INFO: `/v3/users/5a34690403de409ea5bf63b1918f4552` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:13.873084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a34690403de409ea5bf63b1918f4552)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:30:13.874134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:30:13.874134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 21 17:30:13.874134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: Jul 21 17:30:13.875382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:30:13.888139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:13.949841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a34690403de409ea5bf63b1918f4552'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:30:13.950279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a34690403de409ea5bf63b1918f4552'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:30:13.950800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a34690403de409ea5bf63b1918f4552'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:30:13.963539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a34690403de409ea5bf63b1918f4552'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:30:13.983431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a34690403de409ea5bf63b1918f4552'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:30:14.000325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 744/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:30:13 2022] DELETE /identity/v3/users/5a34690403de409ea5bf63b1918f4552 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:30:14.004909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452499c-38fe-45e2-ab38-5a7493362e84 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:14.005305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452499c-38fe-45e2-ab38-5a7493362e84 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:14.005655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452499c-38fe-45e2-ab38-5a7493362e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:14.062320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6452499c-38fe-45e2-ab38-5a7493362e84 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:30:14.063515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6452499c-38fe-45e2-ab38-5a7493362e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:14.142937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:14.157586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 764/1508] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:30:14.166955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:30:14.168211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:14.279501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38094658-7580-4570-9117-9b783efebc8c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:30:14.343021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:14.363877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07e6fb-8baf-424b-878e-2b91a5fc6374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:30:14.364561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07e6fb-8baf-424b-878e-2b91a5fc6374 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:14.364846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07e6fb-8baf-424b-878e-2b91a5fc6374 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:14.365120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07e6fb-8baf-424b-878e-2b91a5fc6374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:14.366754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1f07e6fb-8baf-424b-878e-2b91a5fc6374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:14.475198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07e6fb-8baf-424b-878e-2b91a5fc6374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:30:14.476603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07e6fb-8baf-424b-878e-2b91a5fc6374 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:30:14.495829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 745/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:30:14.864524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452499c-38fe-45e2-ab38-5a7493362e84 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:30:14.865764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6452499c-38fe-45e2-ab38-5a7493362e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:30:14.878510 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452499c-38fe-45e2-ab38-5a7493362e84 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:30:14.986615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:30:14.987464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:30:14.987742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:30:14.988014 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] PATH_INFO: `/v3/projects/d1ce5eb4989d46ee8738500709cf2dca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:30:14.997882 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1ce5eb4989d46ee8738500709cf2dca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:30:14.998867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:30:14.998867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 21 17:30:14.998867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: Jul 21 17:30:15.000060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:30:15.058758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1ce5eb4989d46ee8738500709cf2dca', 'request_id': 'req-9c58a680-fdd1-4219-bfe1-26bbad41199b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:30:15.072671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1ce5eb4989d46ee8738500709cf2dca', 'request_id': 'req-9c58a680-fdd1-4219-bfe1-26bbad41199b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:30:15.082053 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1ce5eb4989d46ee8738500709cf2dca', 'request_id': 'req-9c58a680-fdd1-4219-bfe1-26bbad41199b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:30:15.105380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] The token cache is being invalidate because project d1ce5eb4989d46ee8738500709cf2dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:30:15.105673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:30:15.106191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9c58a680-fdd1-4219-bfe1-26bbad41199b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:30:15.107563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 765/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:30:14 2022] DELETE /identity/v3/projects/d1ce5eb4989d46ee8738500709cf2dca => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:30:24.492600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=62 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:30:25.103302 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:31:54.969524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1920b-7572-40d0-8b95-d6c1279e9e0b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:31:54.970064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1920b-7572-40d0-8b95-d6c1279e9e0b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:31:54.970064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1920b-7572-40d0-8b95-d6c1279e9e0b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:31:54.971478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:31:54.976253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7ee41-4a7a-4f72-ae92-8a3826ce51e2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:31:54.976700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7ee41-4a7a-4f72-ae92-8a3826ce51e2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:31:54.976872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7ee41-4a7a-4f72-ae92-8a3826ce51e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:31:55.056698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ede7ee41-4a7a-4f72-ae92-8a3826ce51e2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:31:55.057912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ede7ee41-4a7a-4f72-ae92-8a3826ce51e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:31:55.161635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:31:55.315967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1920b-7572-40d0-8b95-d6c1279e9e0b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:31:55.318077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e8c1920b-7572-40d0-8b95-d6c1279e9e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:31:55.437353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1920b-7572-40d0-8b95-d6c1279e9e0b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:31:55.442740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d659c-cf8e-4043-87e0-47695954d331 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:31:55.443382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d659c-cf8e-4043-87e0-47695954d331 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:31:55.443665 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d659c-cf8e-4043-87e0-47695954d331 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:31:55.443941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d659c-cf8e-4043-87e0-47695954d331 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:31:55.452313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d659c-cf8e-4043-87e0-47695954d331 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:31:55.453884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d659c-cf8e-4043-87e0-47695954d331 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:31:55.463143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 747/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:31:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:31:56.290205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ba88f-08a3-4ca3-81b0-3aff825dd99f None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:31:56.290783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ba88f-08a3-4ca3-81b0-3aff825dd99f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:31:56.290783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ba88f-08a3-4ca3-81b0-3aff825dd99f None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:31:56.292489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:31:56.297424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18ee7e-dc27-456d-8f65-ea32f09ac34a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:31:56.297873 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18ee7e-dc27-456d-8f65-ea32f09ac34a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:31:56.298050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18ee7e-dc27-456d-8f65-ea32f09ac34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:31:56.315973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7d18ee7e-dc27-456d-8f65-ea32f09ac34a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:31:56.317188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7d18ee7e-dc27-456d-8f65-ea32f09ac34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:31:56.325886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:31:56.480113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ba88f-08a3-4ca3-81b0-3aff825dd99f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:31:56.495640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ba88f-08a3-4ca3-81b0-3aff825dd99f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:31:56.500893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd3b9d-3d84-48c3-9946-841d1453c96e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:31:56.501524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd3b9d-3d84-48c3-9946-841d1453c96e service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:31:56.501805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd3b9d-3d84-48c3-9946-841d1453c96e service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:31:56.502082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd3b9d-3d84-48c3-9946-841d1453c96e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:31:56.510890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd3b9d-3d84-48c3-9946-841d1453c96e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:31:56.512507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd3b9d-3d84-48c3-9946-841d1453c96e service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:31:56.522497 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 768/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:31:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:31:56.551126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678c3b8-bdb0-4fb8-ac0c-9f36baf5ba9d None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:31:56.551768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678c3b8-bdb0-4fb8-ac0c-9f36baf5ba9d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:31:56.551990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678c3b8-bdb0-4fb8-ac0c-9f36baf5ba9d None None] PATH_INFO: `/v3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:31:56.555819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 749/1517] 10.222.0.47 () {54 vars in 925 bytes} [Thu Jul 21 17:31:56 2022] GET /identity/v3 => generated 255 bytes in 6 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:31:56.580946 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50963278-af0a-45fd-bb06-14ca41a9ba8a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:31:56.581341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50963278-af0a-45fd-bb06-14ca41a9ba8a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:31:56.581518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50963278-af0a-45fd-bb06-14ca41a9ba8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:31:56.643814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-50963278-af0a-45fd-bb06-14ca41a9ba8a None None] MFA Rules not processed for user `901cf70064c34b63a0eda979a5ce77e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:31:56.645050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-50963278-af0a-45fd-bb06-14ca41a9ba8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:31:56.664228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:31:56.721258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 769/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jul 21 17:31:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:32:05.468547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7c5de-9a4f-4ce2-a202-d40a0622743a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:05.469066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7c5de-9a4f-4ce2-a202-d40a0622743a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:05.469066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7c5de-9a4f-4ce2-a202-d40a0622743a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:05.470387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 750/1519] 10.222.0.57 () {54 vars in 961 bytes} [Thu Jul 21 17:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:32:05.475280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201634f-deaf-45fe-983c-663a6f223c6b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:05.475704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201634f-deaf-45fe-983c-663a6f223c6b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:05.475836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201634f-deaf-45fe-983c-663a6f223c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:05.494552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a201634f-deaf-45fe-983c-663a6f223c6b None None] MFA Rules not processed for user `901cf70064c34b63a0eda979a5ce77e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:32:05.495714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a201634f-deaf-45fe-983c-663a6f223c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:32:05.504031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 770/1520] 10.222.0.57 () {58 vars in 1058 bytes} [Thu Jul 21 17:32:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:32:05.511306 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7c5de-9a4f-4ce2-a202-d40a0622743a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:32:05.524280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7c5de-9a4f-4ce2-a202-d40a0622743a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:32:05.529811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab2665-f38f-461b-b175-cffc4ed6fcd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:32:05.530466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab2665-f38f-461b-b175-cffc4ed6fcd8 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:05.530748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab2665-f38f-461b-b175-cffc4ed6fcd8 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:05.531016 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab2665-f38f-461b-b175-cffc4ed6fcd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:05.539136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab2665-f38f-461b-b175-cffc4ed6fcd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:32:05.541416 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab2665-f38f-461b-b175-cffc4ed6fcd8 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:32:05.550868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 751/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:32:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:32:07.760983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6aaf-d1e8-41fb-b897-84399900dee1 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:07.761550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6aaf-d1e8-41fb-b897-84399900dee1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:07.761550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6aaf-d1e8-41fb-b897-84399900dee1 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:07.762910 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:32:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:32:07.767760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca37df9-e0ba-4cd3-9d6e-4c90a7e94f8e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:07.768175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca37df9-e0ba-4cd3-9d6e-4c90a7e94f8e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:07.768335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca37df9-e0ba-4cd3-9d6e-4c90a7e94f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:07.785912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-bca37df9-e0ba-4cd3-9d6e-4c90a7e94f8e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:32:07.787097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bca37df9-e0ba-4cd3-9d6e-4c90a7e94f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:32:07.795015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:32:07.949209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b6aaf-d1e8-41fb-b897-84399900dee1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:32:07.963844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b6aaf-d1e8-41fb-b897-84399900dee1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:32:07.969194 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5426fd5-ccbb-4793-ae9f-f2e82077cc7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:32:07.969821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5426fd5-ccbb-4793-ae9f-f2e82077cc7d service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:07.970100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5426fd5-ccbb-4793-ae9f-f2e82077cc7d service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:07.970370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5426fd5-ccbb-4793-ae9f-f2e82077cc7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:07.978680 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5426fd5-ccbb-4793-ae9f-f2e82077cc7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:32:07.980249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5426fd5-ccbb-4793-ae9f-f2e82077cc7d service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:32:07.989434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 772/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:32:08.810965 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08622b-b31c-407a-9007-339b37b92d7a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:08.811549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08622b-b31c-407a-9007-339b37b92d7a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:08.811549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08622b-b31c-407a-9007-339b37b92d7a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:08.812994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 17:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:32:08.818014 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00cb66-9fc6-4238-bef9-f634a15586d7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:08.818455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00cb66-9fc6-4238-bef9-f634a15586d7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:08.818610 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00cb66-9fc6-4238-bef9-f634a15586d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:08.839104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7d00cb66-9fc6-4238-bef9-f634a15586d7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:32:08.840353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7d00cb66-9fc6-4238-bef9-f634a15586d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:32:08.848269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 17:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:32:09.001052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08622b-b31c-407a-9007-339b37b92d7a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:32:09.015952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08622b-b31c-407a-9007-339b37b92d7a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:32:09.021296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80f024-13dc-4359-879a-fe86edb11ecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:32:09.021925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80f024-13dc-4359-879a-fe86edb11ecd service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:09.022209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80f024-13dc-4359-879a-fe86edb11ecd service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:09.022482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80f024-13dc-4359-879a-fe86edb11ecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:09.030984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80f024-13dc-4359-879a-fe86edb11ecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:32:09.032597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80f024-13dc-4359-879a-fe86edb11ecd service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:32:09.041982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 754/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:32:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:32:13.086566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1f284-3579-4d5a-9170-2576e36ed489 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:13.087112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1f284-3579-4d5a-9170-2576e36ed489 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:13.087112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1f284-3579-4d5a-9170-2576e36ed489 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:13.088693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:32:13.093983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb1e4e-fb93-4e2a-b6e9-db35ac3c6669 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:13.094418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb1e4e-fb93-4e2a-b6e9-db35ac3c6669 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:13.094530 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb1e4e-fb93-4e2a-b6e9-db35ac3c6669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:13.112135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-93fb1e4e-fb93-4e2a-b6e9-db35ac3c6669 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:32:13.113333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-93fb1e4e-fb93-4e2a-b6e9-db35ac3c6669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:32:13.121792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:32:13.196375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1f284-3579-4d5a-9170-2576e36ed489 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:32:13.198988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-eda1f284-3579-4d5a-9170-2576e36ed489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:32:13.276713 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1f284-3579-4d5a-9170-2576e36ed489 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:32:13.280260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bc6e9-23d4-48a5-a222-3c5d0961aeeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:32:13.280924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc6e9-23d4-48a5-a222-3c5d0961aeeb service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:13.281203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc6e9-23d4-48a5-a222-3c5d0961aeeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:13.281477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc6e9-23d4-48a5-a222-3c5d0961aeeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:13.290428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bc6e9-23d4-48a5-a222-3c5d0961aeeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:32:13.292411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bc6e9-23d4-48a5-a222-3c5d0961aeeb service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:32:13.301833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 775/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:32:15.336188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e79d91-d9dc-42e7-af71-398eb1454474 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:15.336735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e79d91-d9dc-42e7-af71-398eb1454474 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:15.336735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e79d91-d9dc-42e7-af71-398eb1454474 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:15.338662 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:32:15.344023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fb15d-dd8a-462d-9b9b-d9022cde8d1d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:15.344473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fb15d-dd8a-462d-9b9b-d9022cde8d1d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:15.344622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fb15d-dd8a-462d-9b9b-d9022cde8d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:15.363086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1b4fb15d-dd8a-462d-9b9b-d9022cde8d1d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:32:15.364273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1b4fb15d-dd8a-462d-9b9b-d9022cde8d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:32:15.373494 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:32:15.409567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e79d91-d9dc-42e7-af71-398eb1454474 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:32:15.420896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e79d91-d9dc-42e7-af71-398eb1454474 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:32:15.424414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b37ea-07ae-4c5a-9980-f34fc57cc915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:32:15.425096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b37ea-07ae-4c5a-9980-f34fc57cc915 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:15.425360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b37ea-07ae-4c5a-9980-f34fc57cc915 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:15.425635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b37ea-07ae-4c5a-9980-f34fc57cc915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:15.434563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b37ea-07ae-4c5a-9980-f34fc57cc915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:32:15.435978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b37ea-07ae-4c5a-9980-f34fc57cc915 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:32:15.445376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 757/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:32:18.964382 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b6f1-1d3b-4f91-93a9-2f1220dd9090 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:18.964953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b6f1-1d3b-4f91-93a9-2f1220dd9090 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:18.964953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b6f1-1d3b-4f91-93a9-2f1220dd9090 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:18.966289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:32:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:32:18.971382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7487e-01dd-4ecd-b4b9-74805f575f0a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:18.971822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7487e-01dd-4ecd-b4b9-74805f575f0a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:18.971955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7487e-01dd-4ecd-b4b9-74805f575f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:18.990204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d6d7487e-01dd-4ecd-b4b9-74805f575f0a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:32:18.991400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d6d7487e-01dd-4ecd-b4b9-74805f575f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:32:18.999452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:32:19.304015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b6f1-1d3b-4f91-93a9-2f1220dd9090 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:32:19.305680 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-92e0b6f1-1d3b-4f91-93a9-2f1220dd9090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:32:19.370232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:32:19.383798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b6f1-1d3b-4f91-93a9-2f1220dd9090 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:32:19.387416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524fdd8-ecea-44b5-a741-e2c69ba37089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:32:19.388056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524fdd8-ecea-44b5-a741-e2c69ba37089 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:19.388356 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524fdd8-ecea-44b5-a741-e2c69ba37089 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:19.388626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524fdd8-ecea-44b5-a741-e2c69ba37089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:19.397312 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524fdd8-ecea-44b5-a741-e2c69ba37089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:32:19.398739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524fdd8-ecea-44b5-a741-e2c69ba37089 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:32:19.407975 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 778/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:32:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:32:21.522357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cefc32-6f13-4b5a-9065-a5cfc4c419de None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:21.522837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cefc32-6f13-4b5a-9065-a5cfc4c419de None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:21.522943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cefc32-6f13-4b5a-9065-a5cfc4c419de None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:21.524383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 17:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:32:21.529933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38ef60-6ed6-490a-b47b-509f27ccf80d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:21.530379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38ef60-6ed6-490a-b47b-509f27ccf80d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:21.530487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38ef60-6ed6-490a-b47b-509f27ccf80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:21.549237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-eb38ef60-6ed6-490a-b47b-509f27ccf80d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:32:21.550425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-eb38ef60-6ed6-490a-b47b-509f27ccf80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:32:21.558984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 17:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:32:21.861249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cefc32-6f13-4b5a-9065-a5cfc4c419de None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:32:21.862925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-77cefc32-6f13-4b5a-9065-a5cfc4c419de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:32:21.876748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cefc32-6f13-4b5a-9065-a5cfc4c419de None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:32:21.880228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf04b8-ec4b-471f-bb4b-3bcb3d36ce2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:32:21.880896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf04b8-ec4b-471f-bb4b-3bcb3d36ce2b service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:32:21.881179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf04b8-ec4b-471f-bb4b-3bcb3d36ce2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:32:21.881458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf04b8-ec4b-471f-bb4b-3bcb3d36ce2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:32:21.890542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf04b8-ec4b-471f-bb4b-3bcb3d36ce2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:32:21.891952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf04b8-ec4b-471f-bb4b-3bcb3d36ce2b service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:32:21.901367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 760/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:32:31.544537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=11 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:32:31.897188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:50.636488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed949547-5388-4b59-b2f8-bf80c2a19450 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:50.638299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed949547-5388-4b59-b2f8-bf80c2a19450 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:50.638767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed949547-5388-4b59-b2f8-bf80c2a19450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:50.665336 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ed949547-5388-4b59-b2f8-bf80c2a19450 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:50.666973 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ed949547-5388-4b59-b2f8-bf80c2a19450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:50.679081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 780/1540] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:33:50.690597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8c59e-f9f1-453b-b124-323a566c4fd7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:50.691217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8c59e-f9f1-453b-b124-323a566c4fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:50.691217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8c59e-f9f1-453b-b124-323a566c4fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:50.717650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-59d8c59e-f9f1-453b-b124-323a566c4fd7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:50.718856 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-59d8c59e-f9f1-453b-b124-323a566c4fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:50.729443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 761/1541] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:50.734990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed949547-5388-4b59-b2f8-bf80c2a19450 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:50.750971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed949547-5388-4b59-b2f8-bf80c2a19450 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:50.758478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863fc3e-bb34-4867-ac0f-3242d11fed00 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:50.759184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863fc3e-bb34-4867-ac0f-3242d11fed00 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:50.759759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863fc3e-bb34-4867-ac0f-3242d11fed00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:50.760762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd5401-bc4d-4caa-b3aa-1dd4f93f39ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:50.761442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd5401-bc4d-4caa-b3aa-1dd4f93f39ec admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:50.761729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd5401-bc4d-4caa-b3aa-1dd4f93f39ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:50.762010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd5401-bc4d-4caa-b3aa-1dd4f93f39ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:50.762849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd5401-bc4d-4caa-b3aa-1dd4f93f39ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:33:50.763101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd5401-bc4d-4caa-b3aa-1dd4f93f39ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:50.764756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd5401-bc4d-4caa-b3aa-1dd4f93f39ec admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:50.782068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1863fc3e-bb34-4867-ac0f-3242d11fed00 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:50.783740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1863fc3e-bb34-4867-ac0f-3242d11fed00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:50.785919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 781/1542] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:33:50.794335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd5401-bc4d-4caa-b3aa-1dd4f93f39ec admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:50.798109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 762/1543] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:33:50.807235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d5fa-21c6-43c6-9d22-61ab24194535 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:50.807519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d5fa-21c6-43c6-9d22-61ab24194535 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:50.807977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d5fa-21c6-43c6-9d22-61ab24194535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:50.811141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd5401-bc4d-4caa-b3aa-1dd4f93f39ec admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:50.825933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74f182-4266-45ed-99f6-70796c2719d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:50.826627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74f182-4266-45ed-99f6-70796c2719d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:50.826920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74f182-4266-45ed-99f6-70796c2719d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:50.827200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74f182-4266-45ed-99f6-70796c2719d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:50.827925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74f182-4266-45ed-99f6-70796c2719d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:50.828996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-fc74f182-4266-45ed-99f6-70796c2719d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:33:50.828996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 21 17:33:50.828996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: Jul 21 17:33:50.830218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74f182-4266-45ed-99f6-70796c2719d4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:50.846747 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2651d5fa-21c6-43c6-9d22-61ab24194535 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:50.848143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2651d5fa-21c6-43c6-9d22-61ab24194535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:50.860342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 763/1544] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 21 17:33:50.864806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651d5fa-21c6-43c6-9d22-61ab24194535 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:50.875503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651d5fa-21c6-43c6-9d22-61ab24194535 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:50.889875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2fd54-9829-44b2-8fd9-ec28d67035d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:50.890787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2fd54-9829-44b2-8fd9-ec28d67035d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:50.891237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2fd54-9829-44b2-8fd9-ec28d67035d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:50.891517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2fd54-9829-44b2-8fd9-ec28d67035d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:50.892323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2fd54-9829-44b2-8fd9-ec28d67035d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:33:50.892655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2fd54-9829-44b2-8fd9-ec28d67035d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:50.893737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-b4d2fd54-9829-44b2-8fd9-ec28d67035d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:33:50.893737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 21 17:33:50.893737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: Jul 21 17:33:50.894624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2fd54-9829-44b2-8fd9-ec28d67035d8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:50.908901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 782/1545] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:33:50 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:33:50.912463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 764/1546] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:33:50.917145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74f182-4266-45ed-99f6-70796c2719d4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:50.923705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2fd54-9829-44b2-8fd9-ec28d67035d8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:50.944552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2fd54-9829-44b2-8fd9-ec28d67035d8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.021899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce22328-c0ee-45df-94dd-fdd5c6dacbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.023085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce22328-c0ee-45df-94dd-fdd5c6dacbf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.023085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce22328-c0ee-45df-94dd-fdd5c6dacbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.023085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce22328-c0ee-45df-94dd-fdd5c6dacbf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.023794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce22328-c0ee-45df-94dd-fdd5c6dacbf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.026150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce22328-c0ee-45df-94dd-fdd5c6dacbf9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.063390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 765/1547] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:33:50 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:33:51.065177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280a26-4d6f-41fd-bdfa-2096c7ea3c11 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.065715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280a26-4d6f-41fd-bdfa-2096c7ea3c11 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.066150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74f182-4266-45ed-99f6-70796c2719d4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.066438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280a26-4d6f-41fd-bdfa-2096c7ea3c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.085606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7f280a26-4d6f-41fd-bdfa-2096c7ea3c11 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.087194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7f280a26-4d6f-41fd-bdfa-2096c7ea3c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.137412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9a120-8af7-4a57-942c-a0c621bc93fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.138057 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9a120-8af7-4a57-942c-a0c621bc93fb admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.138354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9a120-8af7-4a57-942c-a0c621bc93fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.138635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9a120-8af7-4a57-942c-a0c621bc93fb admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.139330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9a120-8af7-4a57-942c-a0c621bc93fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.141860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9a120-8af7-4a57-942c-a0c621bc93fb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.160596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 766/1548] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.162766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a37ea-859f-4784-846f-944312629d59 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.163068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a37ea-859f-4784-846f-944312629d59 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.163491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a37ea-859f-4784-846f-944312629d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.175599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 783/1549] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:33:50 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:51.175599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605bb1-3201-4d1d-a592-d3741523be0a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.175599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605bb1-3201-4d1d-a592-d3741523be0a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.175599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605bb1-3201-4d1d-a592-d3741523be0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.183895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-802a37ea-859f-4784-846f-944312629d59 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.185155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-802a37ea-859f-4784-846f-944312629d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.192606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-32605bb1-3201-4d1d-a592-d3741523be0a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.193048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 767/1550] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.193799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-32605bb1-3201-4d1d-a592-d3741523be0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.194260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78182211-4328-416e-b82a-5d6a3292c521 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.194544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78182211-4328-416e-b82a-5d6a3292c521 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.194819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78182211-4328-416e-b82a-5d6a3292c521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.205721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 784/1551] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.206686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32605bb1-3201-4d1d-a592-d3741523be0a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.217771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32605bb1-3201-4d1d-a592-d3741523be0a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.220783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-78182211-4328-416e-b82a-5d6a3292c521 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.221987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-78182211-4328-416e-b82a-5d6a3292c521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.223008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48680976-3eba-4829-9ca3-89fe52b4d5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.223692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48680976-3eba-4829-9ca3-89fe52b4d5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.223982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48680976-3eba-4829-9ca3-89fe52b4d5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.224264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48680976-3eba-4829-9ca3-89fe52b4d5b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.224993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48680976-3eba-4829-9ca3-89fe52b4d5b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.226934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48680976-3eba-4829-9ca3-89fe52b4d5b2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.235325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 768/1552] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.237050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb806-d57f-452f-8278-5f8f54032627 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.237601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb806-d57f-452f-8278-5f8f54032627 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.238080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb806-d57f-452f-8278-5f8f54032627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.250968 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 785/1553] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:51.253726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a5cc4-a413-4cab-8c1c-ce4c98d5f523 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.254685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a5cc4-a413-4cab-8c1c-ce4c98d5f523 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.255138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a5cc4-a413-4cab-8c1c-ce4c98d5f523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.262488 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-01bdb806-d57f-452f-8278-5f8f54032627 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.264615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-01bdb806-d57f-452f-8278-5f8f54032627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.273681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 769/1554] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.274572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ae2a5cc4-a413-4cab-8c1c-ce4c98d5f523 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.275039 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3e08f-a0ad-4859-bfc0-c8750d631510 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.275480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3e08f-a0ad-4859-bfc0-c8750d631510 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.275771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3e08f-a0ad-4859-bfc0-c8750d631510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.275885 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ae2a5cc4-a413-4cab-8c1c-ce4c98d5f523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.286294 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 786/1555] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.287134 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a5cc4-a413-4cab-8c1c-ce4c98d5f523 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.295703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6ed3e08f-a0ad-4859-bfc0-c8750d631510 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.296968 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6ed3e08f-a0ad-4859-bfc0-c8750d631510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.298551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a5cc4-a413-4cab-8c1c-ce4c98d5f523 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.303260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7f8f1-e1b3-48a8-ac65-21d3858ca154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.303924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7f8f1-e1b3-48a8-ac65-21d3858ca154 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.304210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7f8f1-e1b3-48a8-ac65-21d3858ca154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.304520 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7f8f1-e1b3-48a8-ac65-21d3858ca154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.305052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 770/1556] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.305248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7f8f1-e1b3-48a8-ac65-21d3858ca154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.306416 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a25cd-ef6b-444f-919f-31e92727b8e6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.306716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a25cd-ef6b-444f-919f-31e92727b8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.306855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7f8f1-e1b3-48a8-ac65-21d3858ca154 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.307002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a25cd-ef6b-444f-919f-31e92727b8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.318863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 787/1557] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:51.320473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9195-1b0d-48de-81e5-23a1df6216b2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.320473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9195-1b0d-48de-81e5-23a1df6216b2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.320822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9195-1b0d-48de-81e5-23a1df6216b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.329769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-344a25cd-ef6b-444f-919f-31e92727b8e6 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.329990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:51.330953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-344a25cd-ef6b-444f-919f-31e92727b8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.344798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 771/1558] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.345005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-abda9195-1b0d-48de-81e5-23a1df6216b2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.345129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14567c5e-8383-45a1-b868-0071ae6d6bd1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.345259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-abda9195-1b0d-48de-81e5-23a1df6216b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.345367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14567c5e-8383-45a1-b868-0071ae6d6bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.345367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14567c5e-8383-45a1-b868-0071ae6d6bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.357500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 788/1559] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.357500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9195-1b0d-48de-81e5-23a1df6216b2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.366282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-14567c5e-8383-45a1-b868-0071ae6d6bd1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.367477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-14567c5e-8383-45a1-b868-0071ae6d6bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.372834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9195-1b0d-48de-81e5-23a1df6216b2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.380144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680753b9-bbd2-4238-abae-d0483a8767df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.380909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680753b9-bbd2-4238-abae-d0483a8767df admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.381197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680753b9-bbd2-4238-abae-d0483a8767df admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.381476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680753b9-bbd2-4238-abae-d0483a8767df admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.382551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680753b9-bbd2-4238-abae-d0483a8767df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.383215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 772/1560] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.384558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b64c6-109b-4b1f-b243-0001001af41a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.385469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680753b9-bbd2-4238-abae-d0483a8767df admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.386310 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b64c6-109b-4b1f-b243-0001001af41a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.386626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b64c6-109b-4b1f-b243-0001001af41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.402161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 789/1561] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:51.402161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff35ed7-7769-417c-8f7b-97c2835128c7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.402161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff35ed7-7769-417c-8f7b-97c2835128c7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.402161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff35ed7-7769-417c-8f7b-97c2835128c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.408830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-641b64c6-109b-4b1f-b243-0001001af41a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.410077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-641b64c6-109b-4b1f-b243-0001001af41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.421051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8ff35ed7-7769-417c-8f7b-97c2835128c7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:51.421329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 773/1562] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.422525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8ff35ed7-7769-417c-8f7b-97c2835128c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:51.423672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b64c6-109b-4b1f-b243-0001001af41a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.431493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 790/1563] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:51.432923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff35ed7-7769-417c-8f7b-97c2835128c7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.437705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b64c6-109b-4b1f-b243-0001001af41a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.442635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77bf38-edae-4859-9692-b19016187ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.443314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77bf38-edae-4859-9692-b19016187ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.443595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77bf38-edae-4859-9692-b19016187ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.443874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77bf38-edae-4859-9692-b19016187ad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.444656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77bf38-edae-4859-9692-b19016187ad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:33:51.444970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77bf38-edae-4859-9692-b19016187ad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.446182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff35ed7-7769-417c-8f7b-97c2835128c7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.446588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77bf38-edae-4859-9692-b19016187ad9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.448746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:51.452080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b4e5d-2190-427c-87b9-7cd4877abe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.452927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b4e5d-2190-427c-87b9-7cd4877abe4e admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.453215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b4e5d-2190-427c-87b9-7cd4877abe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.453498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b4e5d-2190-427c-87b9-7cd4877abe4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.454240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b4e5d-2190-427c-87b9-7cd4877abe4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.455868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b4e5d-2190-427c-87b9-7cd4877abe4e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.461686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 774/1564] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:33:51.462804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77bf38-edae-4859-9692-b19016187ad9 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.472977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77bf38-edae-4859-9692-b19016187ad9 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.473191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 791/1565] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:51.473925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b4e5d-2190-427c-87b9-7cd4877abe4e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.478306 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15670613-fdfe-4f5e-be38-9421dd725162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.478979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15670613-fdfe-4f5e-be38-9421dd725162 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.479263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15670613-fdfe-4f5e-be38-9421dd725162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.479541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15670613-fdfe-4f5e-be38-9421dd725162 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.480353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15670613-fdfe-4f5e-be38-9421dd725162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:33:51.481346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15670613-fdfe-4f5e-be38-9421dd725162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.483533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15670613-fdfe-4f5e-be38-9421dd725162 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.484897 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b4e5d-2190-427c-87b9-7cd4877abe4e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.490448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306dcd7-3dd0-4ba1-aa91-83359e407809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.492017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306dcd7-3dd0-4ba1-aa91-83359e407809 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.492461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306dcd7-3dd0-4ba1-aa91-83359e407809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.492985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306dcd7-3dd0-4ba1-aa91-83359e407809 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.494524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306dcd7-3dd0-4ba1-aa91-83359e407809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:33:51.495079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306dcd7-3dd0-4ba1-aa91-83359e407809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.497883 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 775/1566] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:33:51.498580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306dcd7-3dd0-4ba1-aa91-83359e407809 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.498856 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15670613-fdfe-4f5e-be38-9421dd725162 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.511426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15670613-fdfe-4f5e-be38-9421dd725162 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.522836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 792/1567] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:33:51.522836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306dcd7-3dd0-4ba1-aa91-83359e407809 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.523130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c322ec4-c18a-439c-b5c4-709bb4e5fbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.523130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c322ec4-c18a-439c-b5c4-709bb4e5fbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.523130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c322ec4-c18a-439c-b5c4-709bb4e5fbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.523130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c322ec4-c18a-439c-b5c4-709bb4e5fbe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.523782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c322ec4-c18a-439c-b5c4-709bb4e5fbe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:33:51.523782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c322ec4-c18a-439c-b5c4-709bb4e5fbe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.523782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c322ec4-c18a-439c-b5c4-709bb4e5fbe7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.534023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306dcd7-3dd0-4ba1-aa91-83359e407809 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.538325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 776/1568] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:33:51.538724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf19aa-4db2-443c-931f-67f36dd2d417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.539399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf19aa-4db2-443c-931f-67f36dd2d417 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.539686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf19aa-4db2-443c-931f-67f36dd2d417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.539966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf19aa-4db2-443c-931f-67f36dd2d417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.540422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c322ec4-c18a-439c-b5c4-709bb4e5fbe7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.541031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf19aa-4db2-443c-931f-67f36dd2d417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.542656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf19aa-4db2-443c-931f-67f36dd2d417 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.554108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 793/1569] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:51.555029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf19aa-4db2-443c-931f-67f36dd2d417 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.555841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c322ec4-c18a-439c-b5c4-709bb4e5fbe7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.563871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fba0c6-e4bb-418b-8de0-e36a48954054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.564592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fba0c6-e4bb-418b-8de0-e36a48954054 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.564882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fba0c6-e4bb-418b-8de0-e36a48954054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.565163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fba0c6-e4bb-418b-8de0-e36a48954054 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.565896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fba0c6-e4bb-418b-8de0-e36a48954054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:33:51.566206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fba0c6-e4bb-418b-8de0-e36a48954054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.566355 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf19aa-4db2-443c-931f-67f36dd2d417 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.567815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fba0c6-e4bb-418b-8de0-e36a48954054 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.571165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2813f4-7a7f-4a06-a5ac-fbcc846ec788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.571855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2813f4-7a7f-4a06-a5ac-fbcc846ec788 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.572168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2813f4-7a7f-4a06-a5ac-fbcc846ec788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.572486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2813f4-7a7f-4a06-a5ac-fbcc846ec788 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.573248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2813f4-7a7f-4a06-a5ac-fbcc846ec788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:33:51.573941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2813f4-7a7f-4a06-a5ac-fbcc846ec788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.575824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2813f4-7a7f-4a06-a5ac-fbcc846ec788 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.582209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 777/1570] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:33:51.582878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fba0c6-e4bb-418b-8de0-e36a48954054 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.590438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 794/1571] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:33:51.591265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2813f4-7a7f-4a06-a5ac-fbcc846ec788 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.596082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fba0c6-e4bb-418b-8de0-e36a48954054 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.601310 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069989d-7bf2-4ef7-8e80-8f6f0d13fd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.601490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2813f4-7a7f-4a06-a5ac-fbcc846ec788 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.602046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069989d-7bf2-4ef7-8e80-8f6f0d13fd2e admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.602270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069989d-7bf2-4ef7-8e80-8f6f0d13fd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.602548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069989d-7bf2-4ef7-8e80-8f6f0d13fd2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.603247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069989d-7bf2-4ef7-8e80-8f6f0d13fd2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.605283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069989d-7bf2-4ef7-8e80-8f6f0d13fd2e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.606853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780d566-dd22-4445-8269-2cad55ad3ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.608659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780d566-dd22-4445-8269-2cad55ad3ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.608659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780d566-dd22-4445-8269-2cad55ad3ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.608659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780d566-dd22-4445-8269-2cad55ad3ecd admin admin] PATH_INFO: `/v3/projects/f52012c88f1e4cc59092d37fcc0fa23a/users/c5e836103d274f9386f2dd933acd982d/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.625527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780d566-dd22-4445-8269-2cad55ad3ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52012c88f1e4cc59092d37fcc0fa23a, user_id=c5e836103d274f9386f2dd933acd982d, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.626993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-8780d566-dd22-4445-8269-2cad55ad3ecd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:33:51.626993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 21 17:33:51.626993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: Jul 21 17:33:51.629351 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780d566-dd22-4445-8269-2cad55ad3ecd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.640961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 778/1572] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:51.642088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069989d-7bf2-4ef7-8e80-8f6f0d13fd2e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.652080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 795/1573] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:51 2022] PUT /identity/v3/projects/f52012c88f1e4cc59092d37fcc0fa23a/users/c5e836103d274f9386f2dd933acd982d/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:51.652080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780d566-dd22-4445-8269-2cad55ad3ecd admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.711000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069989d-7bf2-4ef7-8e80-8f6f0d13fd2e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.722915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780d566-dd22-4445-8269-2cad55ad3ecd admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.727755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf7499-342f-4fde-8d07-9abd22c582e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.728447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf7499-342f-4fde-8d07-9abd22c582e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.728730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf7499-342f-4fde-8d07-9abd22c582e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.729005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf7499-342f-4fde-8d07-9abd22c582e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.729697 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf7499-342f-4fde-8d07-9abd22c582e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.731652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf7499-342f-4fde-8d07-9abd22c582e7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.768380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 796/1574] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:51.768380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf7499-342f-4fde-8d07-9abd22c582e7 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.779317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fb093-f29b-47a4-b34a-6e40845a4384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.780154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fb093-f29b-47a4-b34a-6e40845a4384 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.780478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fb093-f29b-47a4-b34a-6e40845a4384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.780754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fb093-f29b-47a4-b34a-6e40845a4384 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.781836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fb093-f29b-47a4-b34a-6e40845a4384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.783834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fb093-f29b-47a4-b34a-6e40845a4384 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.826322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 779/1575] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/projects => generated 405 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:51.826322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fb093-f29b-47a4-b34a-6e40845a4384 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.827467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:51.841034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf7499-342f-4fde-8d07-9abd22c582e7 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.855477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:51.901225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fb093-f29b-47a4-b34a-6e40845a4384 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:51.905970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69778e0-d1a8-4847-a442-36fd2213feb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.906870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69778e0-d1a8-4847-a442-36fd2213feb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.907158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69778e0-d1a8-4847-a442-36fd2213feb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.907443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69778e0-d1a8-4847-a442-36fd2213feb7 admin admin] PATH_INFO: `/v3/projects/b0917b2a75df4a9ab1d2cbd4fd3f7613/users/fa82bddf9aa0455797d42993fe850ccf/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.921335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8ce20-961b-467c-9688-6bc4d095dc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:51.922454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8ce20-961b-467c-9688-6bc4d095dc56 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:51.922968 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8ce20-961b-467c-9688-6bc4d095dc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:51.923495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8ce20-961b-467c-9688-6bc4d095dc56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:51.924457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8ce20-961b-467c-9688-6bc4d095dc56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.925250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69778e0-d1a8-4847-a442-36fd2213feb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0917b2a75df4a9ab1d2cbd4fd3f7613, user_id=fa82bddf9aa0455797d42993fe850ccf, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:51.927458 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8ce20-961b-467c-9688-6bc4d095dc56 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.927899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69778e0-d1a8-4847-a442-36fd2213feb7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:51.944776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 780/1576] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:51 2022] PUT /identity/v3/projects/b0917b2a75df4a9ab1d2cbd4fd3f7613/users/fa82bddf9aa0455797d42993fe850ccf/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:51.945831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69778e0-d1a8-4847-a442-36fd2213feb7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:51.961526 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 797/1577] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/projects => generated 391 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:51.962264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8ce20-961b-467c-9688-6bc4d095dc56 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.022376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69778e0-d1a8-4847-a442-36fd2213feb7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.034819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8ce20-961b-467c-9688-6bc4d095dc56 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.039739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809cbbc-fcb6-4a8a-a1d3-23d7d46dd233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.040434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809cbbc-fcb6-4a8a-a1d3-23d7d46dd233 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.040722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809cbbc-fcb6-4a8a-a1d3-23d7d46dd233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.040996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809cbbc-fcb6-4a8a-a1d3-23d7d46dd233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.041677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809cbbc-fcb6-4a8a-a1d3-23d7d46dd233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.043943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809cbbc-fcb6-4a8a-a1d3-23d7d46dd233 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.073724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 798/1578] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:52.073724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809cbbc-fcb6-4a8a-a1d3-23d7d46dd233 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.081854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4680b-8772-4f89-a2d8-120cfd6f9e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.082529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4680b-8772-4f89-a2d8-120cfd6f9e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.082819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4680b-8772-4f89-a2d8-120cfd6f9e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.083096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4680b-8772-4f89-a2d8-120cfd6f9e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.083789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4680b-8772-4f89-a2d8-120cfd6f9e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.085792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4680b-8772-4f89-a2d8-120cfd6f9e67 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.111337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 781/1579] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:33:51 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:52.112650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77077081-8eb4-4a30-b689-b9443df3eb90 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.112941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77077081-8eb4-4a30-b689-b9443df3eb90 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.113218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77077081-8eb4-4a30-b689-b9443df3eb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.130868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-77077081-8eb4-4a30-b689-b9443df3eb90 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:52.132078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-77077081-8eb4-4a30-b689-b9443df3eb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:52.136435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809cbbc-fcb6-4a8a-a1d3-23d7d46dd233 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.149557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:52.166247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:52.193891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 782/1580] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:52.194722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77077081-8eb4-4a30-b689-b9443df3eb90 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.200046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18816ade-112b-4c03-ba71-c760c913d56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.200772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18816ade-112b-4c03-ba71-c760c913d56e admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.201700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18816ade-112b-4c03-ba71-c760c913d56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.201991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18816ade-112b-4c03-ba71-c760c913d56e admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.202689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18816ade-112b-4c03-ba71-c760c913d56e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.204668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18816ade-112b-4c03-ba71-c760c913d56e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.205473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77077081-8eb4-4a30-b689-b9443df3eb90 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.210191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a1562-e612-490a-8f59-f02c7b4c37f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.210868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a1562-e612-490a-8f59-f02c7b4c37f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.211157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a1562-e612-490a-8f59-f02c7b4c37f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.211436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a1562-e612-490a-8f59-f02c7b4c37f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.212181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a1562-e612-490a-8f59-f02c7b4c37f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.213177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-e64a1562-e612-490a-8f59-f02c7b4c37f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:33:52.213177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 21 17:33:52.213177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: Jul 21 17:33:52.214388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a1562-e612-490a-8f59-f02c7b4c37f3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.226871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 799/1581] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:33:52 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:52.227727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18816ade-112b-4c03-ba71-c760c913d56e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.240025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 783/1582] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:33:52 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:52.240226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18816ade-112b-4c03-ba71-c760c913d56e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.240377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07c058-6348-4ce5-99b6-29d737913dc9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.240377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07c058-6348-4ce5-99b6-29d737913dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.240377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07c058-6348-4ce5-99b6-29d737913dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.243002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd3331-d419-4860-a14a-d998a60db8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.243656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd3331-d419-4860-a14a-d998a60db8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.243937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd3331-d419-4860-a14a-d998a60db8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.244209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd3331-d419-4860-a14a-d998a60db8c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.244927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd3331-d419-4860-a14a-d998a60db8c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.247525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd3331-d419-4860-a14a-d998a60db8c0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.257661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-de07c058-6348-4ce5-99b6-29d737913dc9 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:52.259117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-de07c058-6348-4ce5-99b6-29d737913dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:52.267554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 784/1583] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:52.271455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 800/1584] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:33:52 2022] POST /identity/v3/users => generated 303 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:52.271725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07c058-6348-4ce5-99b6-29d737913dc9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.271824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd3331-d419-4860-a14a-d998a60db8c0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.280272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07c058-6348-4ce5-99b6-29d737913dc9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.283347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd3331-d419-4860-a14a-d998a60db8c0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.285786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1a060-53bb-4167-8816-9b49b88f241c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.286463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a060-53bb-4167-8816-9b49b88f241c admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.286745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a060-53bb-4167-8816-9b49b88f241c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.287026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a060-53bb-4167-8816-9b49b88f241c admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.287715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1a060-53bb-4167-8816-9b49b88f241c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.288056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa936c9-7024-4d3a-8f65-65547d310330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.288764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa936c9-7024-4d3a-8f65-65547d310330 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.289049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa936c9-7024-4d3a-8f65-65547d310330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.289322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa936c9-7024-4d3a-8f65-65547d310330 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.289892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1a060-53bb-4167-8816-9b49b88f241c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.290011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa936c9-7024-4d3a-8f65-65547d310330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.292002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa936c9-7024-4d3a-8f65-65547d310330 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.311625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 785/1585] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:33:52 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:52.312472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1a060-53bb-4167-8816-9b49b88f241c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.315967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 801/1586] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:33:52 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:52.315967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa936c9-7024-4d3a-8f65-65547d310330 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.322275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1a060-53bb-4167-8816-9b49b88f241c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.327853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fe341-ef5b-4227-8400-5dcae21a6cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.328536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fe341-ef5b-4227-8400-5dcae21a6cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.328819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fe341-ef5b-4227-8400-5dcae21a6cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.329094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fe341-ef5b-4227-8400-5dcae21a6cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.329774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fe341-ef5b-4227-8400-5dcae21a6cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.331661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fe341-ef5b-4227-8400-5dcae21a6cc5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.353350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 786/1587] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:33:52 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:52.354328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fe341-ef5b-4227-8400-5dcae21a6cc5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.364286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fe341-ef5b-4227-8400-5dcae21a6cc5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.368919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc6369-41ce-40eb-a7d7-049cb2feda6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.369580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc6369-41ce-40eb-a7d7-049cb2feda6c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.369863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc6369-41ce-40eb-a7d7-049cb2feda6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.370140 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc6369-41ce-40eb-a7d7-049cb2feda6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.370875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc6369-41ce-40eb-a7d7-049cb2feda6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.372739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc6369-41ce-40eb-a7d7-049cb2feda6c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.384256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa936c9-7024-4d3a-8f65-65547d310330 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.386779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 787/1588] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.390768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc6369-41ce-40eb-a7d7-049cb2feda6c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.390869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b162d-cae7-48b9-a4f9-b94efdfa8b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.390869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b162d-cae7-48b9-a4f9-b94efdfa8b74 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.390869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b162d-cae7-48b9-a4f9-b94efdfa8b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.390869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b162d-cae7-48b9-a4f9-b94efdfa8b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.398173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc6369-41ce-40eb-a7d7-049cb2feda6c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.400653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b162d-cae7-48b9-a4f9-b94efdfa8b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.402100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b162d-cae7-48b9-a4f9-b94efdfa8b74 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.403758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46379e77-a4ff-43b5-9117-99eb49609226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.404448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46379e77-a4ff-43b5-9117-99eb49609226 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.404734 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46379e77-a4ff-43b5-9117-99eb49609226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.405010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46379e77-a4ff-43b5-9117-99eb49609226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.405729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46379e77-a4ff-43b5-9117-99eb49609226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.407306 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46379e77-a4ff-43b5-9117-99eb49609226 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.412598 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 802/1589] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:33:52.413921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b162d-cae7-48b9-a4f9-b94efdfa8b74 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.419056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 788/1590] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.419967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46379e77-a4ff-43b5-9117-99eb49609226 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.424873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b162d-cae7-48b9-a4f9-b94efdfa8b74 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.429468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa947b80-83ea-443e-b7c9-60182976bbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.429674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46379e77-a4ff-43b5-9117-99eb49609226 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.430136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa947b80-83ea-443e-b7c9-60182976bbae admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.430422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa947b80-83ea-443e-b7c9-60182976bbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.431183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa947b80-83ea-443e-b7c9-60182976bbae admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.431427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa947b80-83ea-443e-b7c9-60182976bbae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.433074 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa947b80-83ea-443e-b7c9-60182976bbae admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.433732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02e652-e668-43c8-bab8-146bd1771dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.435246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02e652-e668-43c8-bab8-146bd1771dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.435691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02e652-e668-43c8-bab8-146bd1771dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.436125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02e652-e668-43c8-bab8-146bd1771dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.445278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 803/1591] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.446218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa947b80-83ea-443e-b7c9-60182976bbae admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.449943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02e652-e668-43c8-bab8-146bd1771dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.451498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02e652-e668-43c8-bab8-146bd1771dc7 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.456682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa947b80-83ea-443e-b7c9-60182976bbae admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.462165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 789/1592] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:33:52.462364 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a0c58-2086-45e7-b509-31d0b9380f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.463383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02e652-e668-43c8-bab8-146bd1771dc7 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.463516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a0c58-2086-45e7-b509-31d0b9380f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.463905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a0c58-2086-45e7-b509-31d0b9380f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.464455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a0c58-2086-45e7-b509-31d0b9380f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.465799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a0c58-2086-45e7-b509-31d0b9380f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.467591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a0c58-2086-45e7-b509-31d0b9380f05 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.474256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02e652-e668-43c8-bab8-146bd1771dc7 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.481822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 804/1593] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.481822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a0c58-2086-45e7-b509-31d0b9380f05 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.482974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801e890-e665-4a29-a4b7-55ba8d6d3d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.483765 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801e890-e665-4a29-a4b7-55ba8d6d3d0e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.484053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801e890-e665-4a29-a4b7-55ba8d6d3d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.484340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801e890-e665-4a29-a4b7-55ba8d6d3d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.485076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801e890-e665-4a29-a4b7-55ba8d6d3d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.486707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801e890-e665-4a29-a4b7-55ba8d6d3d0e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.492850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a0c58-2086-45e7-b509-31d0b9380f05 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.499085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afdb9e-9c0f-4b0c-ae8c-037bf1eac642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.503711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afdb9e-9c0f-4b0c-ae8c-037bf1eac642 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.503913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 790/1594] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.504030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afdb9e-9c0f-4b0c-ae8c-037bf1eac642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.504030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afdb9e-9c0f-4b0c-ae8c-037bf1eac642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.504231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801e890-e665-4a29-a4b7-55ba8d6d3d0e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.504374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afdb9e-9c0f-4b0c-ae8c-037bf1eac642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.504374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afdb9e-9c0f-4b0c-ae8c-037bf1eac642 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.516020 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801e890-e665-4a29-a4b7-55ba8d6d3d0e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.516374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 805/1595] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.517206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afdb9e-9c0f-4b0c-ae8c-037bf1eac642 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.522238 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bdac2-b3ee-4865-be14-6f7e9f2ba3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.523759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bdac2-b3ee-4865-be14-6f7e9f2ba3ff admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.523759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bdac2-b3ee-4865-be14-6f7e9f2ba3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.523971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bdac2-b3ee-4865-be14-6f7e9f2ba3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.524950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bdac2-b3ee-4865-be14-6f7e9f2ba3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.526727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bdac2-b3ee-4865-be14-6f7e9f2ba3ff admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.527468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afdb9e-9c0f-4b0c-ae8c-037bf1eac642 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.533475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd218cca-cadd-48fe-a752-cf80ac64ea0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.534115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd218cca-cadd-48fe-a752-cf80ac64ea0a admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.534398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd218cca-cadd-48fe-a752-cf80ac64ea0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.534886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd218cca-cadd-48fe-a752-cf80ac64ea0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.535396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd218cca-cadd-48fe-a752-cf80ac64ea0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.537079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd218cca-cadd-48fe-a752-cf80ac64ea0a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.544884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 791/1596] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.546290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bdac2-b3ee-4865-be14-6f7e9f2ba3ff admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.548993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 806/1597] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.552569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd218cca-cadd-48fe-a752-cf80ac64ea0a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.558753 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bdac2-b3ee-4865-be14-6f7e9f2ba3ff admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.563465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd218cca-cadd-48fe-a752-cf80ac64ea0a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.565349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fed9ec-8aae-4e9b-9842-3a7a273a33dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.566125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fed9ec-8aae-4e9b-9842-3a7a273a33dc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.566516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fed9ec-8aae-4e9b-9842-3a7a273a33dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.566895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fed9ec-8aae-4e9b-9842-3a7a273a33dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.567862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fed9ec-8aae-4e9b-9842-3a7a273a33dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.568842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4ba6e-22f6-4f5b-9cef-99a27b57a7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.569509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4ba6e-22f6-4f5b-9cef-99a27b57a7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.569784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fed9ec-8aae-4e9b-9842-3a7a273a33dc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.569915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4ba6e-22f6-4f5b-9cef-99a27b57a7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.570060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4ba6e-22f6-4f5b-9cef-99a27b57a7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.570788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4ba6e-22f6-4f5b-9cef-99a27b57a7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.572682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4ba6e-22f6-4f5b-9cef-99a27b57a7b9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.583101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 792/1598] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.587396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 807/1599] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.587396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4ba6e-22f6-4f5b-9cef-99a27b57a7b9 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.587645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fed9ec-8aae-4e9b-9842-3a7a273a33dc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.598562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4ba6e-22f6-4f5b-9cef-99a27b57a7b9 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.601526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fed9ec-8aae-4e9b-9842-3a7a273a33dc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.604344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b06b42-634d-413e-a1ac-0bb19686e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.605304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b06b42-634d-413e-a1ac-0bb19686e7be admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.605673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b06b42-634d-413e-a1ac-0bb19686e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.606641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b06b42-634d-413e-a1ac-0bb19686e7be admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.607870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b06b42-634d-413e-a1ac-0bb19686e7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.608474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bcfa4-1317-4949-b893-442f7d22f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.609153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bcfa4-1317-4949-b893-442f7d22f9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.609431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bcfa4-1317-4949-b893-442f7d22f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.609713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bcfa4-1317-4949-b893-442f7d22f9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.610211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b06b42-634d-413e-a1ac-0bb19686e7be admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.610429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bcfa4-1317-4949-b893-442f7d22f9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.612183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bcfa4-1317-4949-b893-442f7d22f9ee admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.622254 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 808/1600] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.623465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b06b42-634d-413e-a1ac-0bb19686e7be admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.626411 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 793/1601] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:52.626575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bcfa4-1317-4949-b893-442f7d22f9ee admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.634663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b06b42-634d-413e-a1ac-0bb19686e7be admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.640506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bcfa4-1317-4949-b893-442f7d22f9ee admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.641565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ee3f3-ef2c-430e-b038-d8c666521d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.642614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ee3f3-ef2c-430e-b038-d8c666521d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.643204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ee3f3-ef2c-430e-b038-d8c666521d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.643204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ee3f3-ef2c-430e-b038-d8c666521d95 admin admin] PATH_INFO: `/v3/projects/29e74e3f52ac4b6eb555f458eff85297/users/0202ce3a9ad3475892f7115bf9ff7567/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.646993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:52.647932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edecfe-167b-46dc-acb8-5bbe8129a385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.649122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edecfe-167b-46dc-acb8-5bbe8129a385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.649955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edecfe-167b-46dc-acb8-5bbe8129a385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.650244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edecfe-167b-46dc-acb8-5bbe8129a385 admin admin] PATH_INFO: `/v3/projects/1fd3b826ab3d4dfe977c098f0071762d/users/e940d6ceaa6d48be8dbfe4c6ce6f6238/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.663321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ee3f3-ef2c-430e-b038-d8c666521d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e74e3f52ac4b6eb555f458eff85297, user_id=0202ce3a9ad3475892f7115bf9ff7567, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.666052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ee3f3-ef2c-430e-b038-d8c666521d95 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.671775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edecfe-167b-46dc-acb8-5bbe8129a385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd3b826ab3d4dfe977c098f0071762d, user_id=e940d6ceaa6d48be8dbfe4c6ce6f6238, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.675580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edecfe-167b-46dc-acb8-5bbe8129a385 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.682749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 809/1602] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:52 2022] PUT /identity/v3/projects/29e74e3f52ac4b6eb555f458eff85297/users/0202ce3a9ad3475892f7115bf9ff7567/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:52.683583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ee3f3-ef2c-430e-b038-d8c666521d95 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.697054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 794/1603] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:52 2022] PUT /identity/v3/projects/1fd3b826ab3d4dfe977c098f0071762d/users/e940d6ceaa6d48be8dbfe4c6ce6f6238/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:52.698168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edecfe-167b-46dc-acb8-5bbe8129a385 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.744875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:52.769722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edecfe-167b-46dc-acb8-5bbe8129a385 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.775508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203f15c-11b6-4c1b-aabd-2221786c728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.776496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203f15c-11b6-4c1b-aabd-2221786c728e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.776852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203f15c-11b6-4c1b-aabd-2221786c728e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.777198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203f15c-11b6-4c1b-aabd-2221786c728e admin admin] PATH_INFO: `/v3/projects/21c11b3abe754d86919bbcf31ecb312a/users/1bf6e7a4afaf405caae7d750973d9956/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.784410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ee3f3-ef2c-430e-b038-d8c666521d95 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.789269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8527cd-3822-4e49-ac1a-a6fe2d633523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.790344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8527cd-3822-4e49-ac1a-a6fe2d633523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.790653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8527cd-3822-4e49-ac1a-a6fe2d633523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.790990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8527cd-3822-4e49-ac1a-a6fe2d633523 admin admin] PATH_INFO: `/v3/projects/e21cd5e8f00a43f4a32eb6390b585b59/users/83e8fe8d447840dea4510dccf5755a15/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.797000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203f15c-11b6-4c1b-aabd-2221786c728e admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c11b3abe754d86919bbcf31ecb312a, user_id=1bf6e7a4afaf405caae7d750973d9956, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.801821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203f15c-11b6-4c1b-aabd-2221786c728e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.813965 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8527cd-3822-4e49-ac1a-a6fe2d633523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e21cd5e8f00a43f4a32eb6390b585b59, user_id=83e8fe8d447840dea4510dccf5755a15, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.817269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8527cd-3822-4e49-ac1a-a6fe2d633523 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.819984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 795/1604] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:52 2022] PUT /identity/v3/projects/21c11b3abe754d86919bbcf31ecb312a/users/1bf6e7a4afaf405caae7d750973d9956/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:52.821566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203f15c-11b6-4c1b-aabd-2221786c728e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.836234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 810/1605] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:52 2022] PUT /identity/v3/projects/e21cd5e8f00a43f4a32eb6390b585b59/users/83e8fe8d447840dea4510dccf5755a15/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:52.837483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8527cd-3822-4e49-ac1a-a6fe2d633523 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:52.898872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203f15c-11b6-4c1b-aabd-2221786c728e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.910678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8527cd-3822-4e49-ac1a-a6fe2d633523 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:52.920268 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2af18-37f9-446d-b061-1f813cc740ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.921294 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2af18-37f9-446d-b061-1f813cc740ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.921873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2af18-37f9-446d-b061-1f813cc740ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.922034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2af18-37f9-446d-b061-1f813cc740ed admin admin] PATH_INFO: `/v3/projects/10bcea77b2814563943c60926660f4f8/users/8105d7426551465dbefd1b339ec915b1/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.950439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2af18-37f9-446d-b061-1f813cc740ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=10bcea77b2814563943c60926660f4f8, user_id=8105d7426551465dbefd1b339ec915b1, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:52.955039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2af18-37f9-446d-b061-1f813cc740ed admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:52.979136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 811/1606] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:52 2022] PUT /identity/v3/projects/10bcea77b2814563943c60926660f4f8/users/8105d7426551465dbefd1b339ec915b1/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:52.981436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6cff3-5aae-47e1-a79a-b52751393569 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.981832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6cff3-5aae-47e1-a79a-b52751393569 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.982138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6cff3-5aae-47e1-a79a-b52751393569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:52.997629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dfd18-a7a5-4330-a455-76b77947412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:52.998440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dfd18-a7a5-4330-a455-76b77947412b admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:52.998712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dfd18-a7a5-4330-a455-76b77947412b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:52.999017 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dfd18-a7a5-4330-a455-76b77947412b admin admin] PATH_INFO: `/v3/projects/5161720524a4488aa42e383597d30144/users/2ea3ea36439143b8a55f181170efc14a/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.014444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-edf6cff3-5aae-47e1-a79a-b52751393569 None None] MFA Rules not processed for user `0202ce3a9ad3475892f7115bf9ff7567`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:53.016215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-edf6cff3-5aae-47e1-a79a-b52751393569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:53.025345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dfd18-a7a5-4330-a455-76b77947412b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5161720524a4488aa42e383597d30144, user_id=2ea3ea36439143b8a55f181170efc14a, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:53.030661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dfd18-a7a5-4330-a455-76b77947412b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:53.055205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 796/1607] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:52 2022] PUT /identity/v3/projects/5161720524a4488aa42e383597d30144/users/2ea3ea36439143b8a55f181170efc14a/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:53.055205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1fb30-9bf8-45b9-b21a-01a3805d85f5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.055205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1fb30-9bf8-45b9-b21a-01a3805d85f5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.055205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1fb30-9bf8-45b9-b21a-01a3805d85f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.094296 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:53.121165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8ce1fb30-9bf8-45b9-b21a-01a3805d85f5 None None] MFA Rules not processed for user `e940d6ceaa6d48be8dbfe4c6ce6f6238`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:53.123472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8ce1fb30-9bf8-45b9-b21a-01a3805d85f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:53.130485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 812/1608] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:53.131946 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a67d7d-7b46-4ccb-a69c-c8865c23e144 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.132239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a67d7d-7b46-4ccb-a69c-c8865c23e144 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.132883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a67d7d-7b46-4ccb-a69c-c8865c23e144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.193202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:53.230553 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 797/1609] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:53.233344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af25df9-5b0a-4397-92d0-f973fa8e6f0a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.233828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af25df9-5b0a-4397-92d0-f973fa8e6f0a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.234236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af25df9-5b0a-4397-92d0-f973fa8e6f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.256903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3af25df9-5b0a-4397-92d0-f973fa8e6f0a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:53.258298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3af25df9-5b0a-4397-92d0-f973fa8e6f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:53.320337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b9a67d7d-7b46-4ccb-a69c-c8865c23e144 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:53.322066 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b9a67d7d-7b46-4ccb-a69c-c8865c23e144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:53.345339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 798/1610] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:53.348709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28fce1-9258-4c33-935c-d43b64e11c4a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.348709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28fce1-9258-4c33-935c-d43b64e11c4a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.348709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28fce1-9258-4c33-935c-d43b64e11c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.383402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ce28fce1-9258-4c33-935c-d43b64e11c4a None None] MFA Rules not processed for user `8105d7426551465dbefd1b339ec915b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:53.384941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ce28fce1-9258-4c33-935c-d43b64e11c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:53.406531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 813/1611] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:53.408318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463d252-0691-4276-adad-eff179430e0e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.408830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463d252-0691-4276-adad-eff179430e0e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.409323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463d252-0691-4276-adad-eff179430e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.434960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e463d252-0691-4276-adad-eff179430e0e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:53.436329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e463d252-0691-4276-adad-eff179430e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:53.447485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 814/1612] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:53.447485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463d252-0691-4276-adad-eff179430e0e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:53.467899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 799/1613] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:53.469581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a0ed3-59bb-4acb-b27d-c91cdb9a7917 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.470036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a0ed3-59bb-4acb-b27d-c91cdb9a7917 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.470453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a0ed3-59bb-4acb-b27d-c91cdb9a7917 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.472530 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 800/1614] 10.222.0.57 () {54 vars in 962 bytes} [Thu Jul 21 17:33:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:33:53.473490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a0ed3-59bb-4acb-b27d-c91cdb9a7917 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:53.514413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463d252-0691-4276-adad-eff179430e0e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:53.518426 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb97f80-25e1-4c92-a3a4-7952712789b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:53.519079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb97f80-25e1-4c92-a3a4-7952712789b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.519360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb97f80-25e1-4c92-a3a4-7952712789b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.519641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb97f80-25e1-4c92-a3a4-7952712789b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.542658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a0ed3-59bb-4acb-b27d-c91cdb9a7917 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:53.550633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aac38c-6d33-464c-a0b4-587ec2bb177f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:53.551153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aac38c-6d33-464c-a0b4-587ec2bb177f service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.551774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aac38c-6d33-464c-a0b4-587ec2bb177f service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.552062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aac38c-6d33-464c-a0b4-587ec2bb177f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.566829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aac38c-6d33-464c-a0b4-587ec2bb177f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:53.569593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aac38c-6d33-464c-a0b4-587ec2bb177f service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:53.600867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 801/1615] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:33:53.601932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aac38c-6d33-464c-a0b4-587ec2bb177f service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:53.612107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb97f80-25e1-4c92-a3a4-7952712789b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:53.613863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb97f80-25e1-4c92-a3a4-7952712789b3 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:53.614797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aac38c-6d33-464c-a0b4-587ec2bb177f service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:53.619394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5597ca0-489d-4751-9954-81339d07ed47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:53.620157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5597ca0-489d-4751-9954-81339d07ed47 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.620646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5597ca0-489d-4751-9954-81339d07ed47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.621141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5597ca0-489d-4751-9954-81339d07ed47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.633822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5597ca0-489d-4751-9954-81339d07ed47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:53.635618 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5597ca0-489d-4751-9954-81339d07ed47 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:53.640527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 815/1616] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:33:53.643308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:53.645610 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb97f80-25e1-4c92-a3a4-7952712789b3 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:53.650328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 802/1617] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:33:53.651202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5597ca0-489d-4751-9954-81339d07ed47 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:53.663991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5597ca0-489d-4751-9954-81339d07ed47 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:53.667216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb97f80-25e1-4c92-a3a4-7952712789b3 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:53.674744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5ec11-4c90-48fe-ad4f-995a63cfb19c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:53.675591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f57236-bfae-4c74-9160-5176d50c2723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:53.676790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f57236-bfae-4c74-9160-5176d50c2723 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.677439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5ec11-4c90-48fe-ad4f-995a63cfb19c service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.677979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f57236-bfae-4c74-9160-5176d50c2723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.678397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5ec11-4c90-48fe-ad4f-995a63cfb19c service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.678616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5ec11-4c90-48fe-ad4f-995a63cfb19c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.678733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f57236-bfae-4c74-9160-5176d50c2723 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.680953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f57236-bfae-4c74-9160-5176d50c2723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:53.683977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f57236-bfae-4c74-9160-5176d50c2723 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:53.710251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5ec11-4c90-48fe-ad4f-995a63cfb19c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:53.713004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5ec11-4c90-48fe-ad4f-995a63cfb19c service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:53.731836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 803/1618] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:33:53 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:53.733780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a8b25-2f61-4c34-9bb0-d6d37a0bda53 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.734327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a8b25-2f61-4c34-9bb0-d6d37a0bda53 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.734881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a8b25-2f61-4c34-9bb0-d6d37a0bda53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.744706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 816/1619] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:33:53.747921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5ec11-4c90-48fe-ad4f-995a63cfb19c service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:53.768903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5ec11-4c90-48fe-ad4f-995a63cfb19c service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:53.796202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:53.796979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d10a8b25-2f61-4c34-9bb0-d6d37a0bda53 None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:53.798959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d10a8b25-2f61-4c34-9bb0-d6d37a0bda53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:53.914002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883cd21-19ae-4389-baa3-67cf5cfba0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:53.914712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883cd21-19ae-4389-baa3-67cf5cfba0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.915002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883cd21-19ae-4389-baa3-67cf5cfba0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.915283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883cd21-19ae-4389-baa3-67cf5cfba0e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.915995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883cd21-19ae-4389-baa3-67cf5cfba0e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:53.926284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883cd21-19ae-4389-baa3-67cf5cfba0e5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:53.930083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 804/1620] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Jul 21 17:33:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:53.937967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a8b25-2f61-4c34-9bb0-d6d37a0bda53 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:53.971014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a8b25-2f61-4c34-9bb0-d6d37a0bda53 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:53.978190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 817/1621] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:33:53 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:53.979175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883cd21-19ae-4389-baa3-67cf5cfba0e5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:53.989563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e4e9d-26d6-4d63-aa55-77d50baa01dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:53.990618 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e4e9d-26d6-4d63-aa55-77d50baa01dd service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:53.990618 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e4e9d-26d6-4d63-aa55-77d50baa01dd service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:53.990994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e4e9d-26d6-4d63-aa55-77d50baa01dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:53.999203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883cd21-19ae-4389-baa3-67cf5cfba0e5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:54.021710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0f25b-b84f-4a8e-93fb-81ca41c05c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:54.022340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0f25b-b84f-4a8e-93fb-81ca41c05c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:54.022630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0f25b-b84f-4a8e-93fb-81ca41c05c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:54.022914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0f25b-b84f-4a8e-93fb-81ca41c05c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:54.023663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0f25b-b84f-4a8e-93fb-81ca41c05c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:54.031567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0f25b-b84f-4a8e-93fb-81ca41c05c61 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:54.061451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 818/1622] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:54.062559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0f25b-b84f-4a8e-93fb-81ca41c05c61 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:54.081544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0f25b-b84f-4a8e-93fb-81ca41c05c61 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:54.095550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7013e-88b9-4da6-8970-45d1d5fed34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:54.098340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7013e-88b9-4da6-8970-45d1d5fed34e admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:54.098703 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7013e-88b9-4da6-8970-45d1d5fed34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:54.099002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7013e-88b9-4da6-8970-45d1d5fed34e admin admin] PATH_INFO: `/v3/projects/5ad49eae079d491f8b27129553c0f628/users/2abcd1bfb9834cc3b5873933cc35da73/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:54.137998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7013e-88b9-4da6-8970-45d1d5fed34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad49eae079d491f8b27129553c0f628, user_id=2abcd1bfb9834cc3b5873933cc35da73, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:54.144276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7013e-88b9-4da6-8970-45d1d5fed34e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:54.175672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 819/1623] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:54 2022] PUT /identity/v3/projects/5ad49eae079d491f8b27129553c0f628/users/2abcd1bfb9834cc3b5873933cc35da73/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:54.179682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7013e-88b9-4da6-8970-45d1d5fed34e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:54.185502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e4e9d-26d6-4d63-aa55-77d50baa01dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:54.191329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e4e9d-26d6-4d63-aa55-77d50baa01dd service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:54.225049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 805/1624] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:33:54.381111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7013e-88b9-4da6-8970-45d1d5fed34e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:54.394643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c96ed-0acb-404d-bff3-d4328d3f27a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:54.396055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c96ed-0acb-404d-bff3-d4328d3f27a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:54.398201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c96ed-0acb-404d-bff3-d4328d3f27a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:54.398702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c96ed-0acb-404d-bff3-d4328d3f27a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:54.400420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c96ed-0acb-404d-bff3-d4328d3f27a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:54.406589 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c96ed-0acb-404d-bff3-d4328d3f27a5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:54.433149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 820/1625] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:33:54.435063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e4e9d-26d6-4d63-aa55-77d50baa01dd service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:54.449295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e4e9d-26d6-4d63-aa55-77d50baa01dd service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:54.462844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50653f-6fcd-44c7-beac-81c2fa030605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:54.464155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50653f-6fcd-44c7-beac-81c2fa030605 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:54.466249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50653f-6fcd-44c7-beac-81c2fa030605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:54.466939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50653f-6fcd-44c7-beac-81c2fa030605 admin admin] PATH_INFO: `/v3/projects/5ad49eae079d491f8b27129553c0f628/users/2abcd1bfb9834cc3b5873933cc35da73/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:54.469314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50653f-6fcd-44c7-beac-81c2fa030605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad49eae079d491f8b27129553c0f628, user_id=2abcd1bfb9834cc3b5873933cc35da73, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:54.472242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50653f-6fcd-44c7-beac-81c2fa030605 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:54.496083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 806/1626] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:54 2022] PUT /identity/v3/projects/5ad49eae079d491f8b27129553c0f628/users/2abcd1bfb9834cc3b5873933cc35da73/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:54.497860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f447-1f88-4afb-b09f-1f639ab8c46f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:54.498411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f447-1f88-4afb-b09f-1f639ab8c46f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:54.498781 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f447-1f88-4afb-b09f-1f639ab8c46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:54.550050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b617f447-1f88-4afb-b09f-1f639ab8c46f None None] MFA Rules not processed for user `2abcd1bfb9834cc3b5873933cc35da73`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:54.551207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b617f447-1f88-4afb-b09f-1f639ab8c46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:54.656885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5b941-41c9-401d-95e9-8a7eb2072a45 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:54.657415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5b941-41c9-401d-95e9-8a7eb2072a45 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:54.657862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5b941-41c9-401d-95e9-8a7eb2072a45 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:54.659428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 807/1627] 10.222.0.57 () {54 vars in 912 bytes} [Thu Jul 21 17:33:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:33:54.665296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f24981-8f10-4005-b806-84b5b6b9a8df None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:54.665930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f24981-8f10-4005-b806-84b5b6b9a8df None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:54.666358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f24981-8f10-4005-b806-84b5b6b9a8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:54.689508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:54.712991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b3f24981-8f10-4005-b806-84b5b6b9a8df None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:54.714597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b3f24981-8f10-4005-b806-84b5b6b9a8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:54.734802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 821/1628] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:54 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:33:54.747152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617f447-1f88-4afb-b09f-1f639ab8c46f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:54.844042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617f447-1f88-4afb-b09f-1f639ab8c46f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:54.844923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:54.851084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f712f8-18f9-425d-92e4-9544c7806035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:54.851760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f712f8-18f9-425d-92e4-9544c7806035 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:54.853381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f712f8-18f9-425d-92e4-9544c7806035 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:54.853693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f712f8-18f9-425d-92e4-9544c7806035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:54.870685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f712f8-18f9-425d-92e4-9544c7806035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:54.873742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f712f8-18f9-425d-92e4-9544c7806035 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:54.901626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 822/1629] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:33:54.960244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 808/1630] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 17:33:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:33:54.988824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f712f8-18f9-425d-92e4-9544c7806035 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:54.998614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f712f8-18f9-425d-92e4-9544c7806035 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:55.009365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de2ad-0354-42c7-8396-65044a59a670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:55.012640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de2ad-0354-42c7-8396-65044a59a670 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:55.013029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de2ad-0354-42c7-8396-65044a59a670 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:55.013199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de2ad-0354-42c7-8396-65044a59a670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:55.039994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19de2ad-0354-42c7-8396-65044a59a670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:55.044482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19de2ad-0354-42c7-8396-65044a59a670 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:55.071515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 823/1631] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:33:55.818989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f24981-8f10-4005-b806-84b5b6b9a8df None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:56.037924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f24981-8f10-4005-b806-84b5b6b9a8df None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:56.051707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:56.053336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:56.055906 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:56.056762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] PATH_INFO: `/v3/users/8105d7426551465dbefd1b339ec915b1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:56.064465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8105d7426551465dbefd1b339ec915b1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:56.069316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:56.289022 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8105d7426551465dbefd1b339ec915b1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:56.290892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8105d7426551465dbefd1b339ec915b1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:56.291902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8105d7426551465dbefd1b339ec915b1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:56.316649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8105d7426551465dbefd1b339ec915b1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:56.345259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c6a0bc7f-41ff-467b-8d9d-cfadf13c7686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8105d7426551465dbefd1b339ec915b1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:56.374389 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 809/1632] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:33:55 2022] DELETE /identity/v3/users/8105d7426551465dbefd1b339ec915b1 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:56.379300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba36c9-c2a0-4704-8bf2-5275fe63887e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:56.380422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba36c9-c2a0-4704-8bf2-5275fe63887e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:56.380852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba36c9-c2a0-4704-8bf2-5275fe63887e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:56.424173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-33ba36c9-c2a0-4704-8bf2-5275fe63887e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:56.429284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-33ba36c9-c2a0-4704-8bf2-5275fe63887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:56.567110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa6e1d-0ce0-43cd-972d-1bb3e0e73bdb None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:56.569620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa6e1d-0ce0-43cd-972d-1bb3e0e73bdb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:56.569932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa6e1d-0ce0-43cd-972d-1bb3e0e73bdb None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:56.572520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 810/1633] 10.222.0.57 () {54 vars in 962 bytes} [Thu Jul 21 17:33:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:33:56.583466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba3a00-bef6-4e61-bd00-81920837408f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:56.583752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba3a00-bef6-4e61-bd00-81920837408f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:56.584189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba3a00-bef6-4e61-bd00-81920837408f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:56.605925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 824/1634] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:56.625078 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba36c9-c2a0-4704-8bf2-5275fe63887e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:56.626873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-33ba36c9-c2a0-4704-8bf2-5275fe63887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:56.633359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-38ba3a00-bef6-4e61-bd00-81920837408f None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:56.634867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-38ba3a00-bef6-4e61-bd00-81920837408f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:56.736812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:56.774260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba36c9-c2a0-4704-8bf2-5275fe63887e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:56.778876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb398f-9d3d-44fa-aae5-91056e68f88b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:56.779881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb398f-9d3d-44fa-aae5-91056e68f88b service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:56.780260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb398f-9d3d-44fa-aae5-91056e68f88b service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:56.780657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb398f-9d3d-44fa-aae5-91056e68f88b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:56.798455 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 811/1635] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Jul 21 17:33:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:33:56.801366 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb398f-9d3d-44fa-aae5-91056e68f88b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:56.807467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb398f-9d3d-44fa-aae5-91056e68f88b service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:56.807973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546140c-9bad-48b7-a1e1-29c895609769 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:56.808590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546140c-9bad-48b7-a1e1-29c895609769 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:56.809732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546140c-9bad-48b7-a1e1-29c895609769 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:56.815871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 812/1636] 10.222.0.57 () {54 vars in 912 bytes} [Thu Jul 21 17:33:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:33:56.821768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dfd58-4885-40e6-a38f-a060e9cd5785 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:56.822414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dfd58-4885-40e6-a38f-a060e9cd5785 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:56.822879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dfd58-4885-40e6-a38f-a060e9cd5785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:56.835327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 825/1637] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:33:56.862022 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-155dfd58-4885-40e6-a38f-a060e9cd5785 None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:56.863269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-155dfd58-4885-40e6-a38f-a060e9cd5785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:56.895628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 813/1638] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 17:33:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:33:56.903405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb398f-9d3d-44fa-aae5-91056e68f88b service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:56.905007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7ffb398f-9d3d-44fa-aae5-91056e68f88b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:56.978539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:57.044821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb398f-9d3d-44fa-aae5-91056e68f88b service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:57.049143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b0a3a-0731-403e-8e93-a5c330050b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:57.050111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b0a3a-0731-403e-8e93-a5c330050b2c service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:57.050476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b0a3a-0731-403e-8e93-a5c330050b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:57.050847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b0a3a-0731-403e-8e93-a5c330050b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:57.063732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b0a3a-0731-403e-8e93-a5c330050b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:57.065498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b0a3a-0731-403e-8e93-a5c330050b2c service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:57.092118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 826/1639] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:33:57.295182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d9961-5b14-4142-8daa-bf80e7acfbdb None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:57.300783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d9961-5b14-4142-8daa-bf80e7acfbdb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:57.301268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d9961-5b14-4142-8daa-bf80e7acfbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:57.349767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-514d9961-5b14-4142-8daa-bf80e7acfbdb None None] MFA Rules not processed for user `83e8fe8d447840dea4510dccf5755a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:57.351398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-514d9961-5b14-4142-8daa-bf80e7acfbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:57.550680 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 814/1640] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:57 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:57.552170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b0a3a-0731-403e-8e93-a5c330050b2c service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:57.557931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3d5b0a3a-0731-403e-8e93-a5c330050b2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:57.595665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b0a3a-0731-403e-8e93-a5c330050b2c service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:57.616214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c273c4d-d3c1-451b-a786-9e4233f8b787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:57.616923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c273c4d-d3c1-451b-a786-9e4233f8b787 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:57.627946 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c273c4d-d3c1-451b-a786-9e4233f8b787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:57.628240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c273c4d-d3c1-451b-a786-9e4233f8b787 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:57.630377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c273c4d-d3c1-451b-a786-9e4233f8b787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:57.632333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c273c4d-d3c1-451b-a786-9e4233f8b787 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:57.701672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 827/1641] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:33:57 2022] POST /identity/v3/projects => generated 405 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:57.706533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d9961-5b14-4142-8daa-bf80e7acfbdb None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:57.720838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d9961-5b14-4142-8daa-bf80e7acfbdb None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:57.903648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4b8a4-f44d-4bce-9e9d-70f83680cb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:57.905588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4b8a4-f44d-4bce-9e9d-70f83680cb69 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:57.907129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4b8a4-f44d-4bce-9e9d-70f83680cb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:57.908124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4b8a4-f44d-4bce-9e9d-70f83680cb69 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:57.909096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4b8a4-f44d-4bce-9e9d-70f83680cb69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:57.911341 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4b8a4-f44d-4bce-9e9d-70f83680cb69 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:57.941844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 815/1642] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:33:57 2022] POST /identity/v3/users => generated 309 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:33:57.946488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c273c4d-d3c1-451b-a786-9e4233f8b787 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:57.986642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c273c4d-d3c1-451b-a786-9e4233f8b787 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:57.993256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640a8e1-aabd-4d72-b941-b77d9b5d7867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:57.994512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a8e1-aabd-4d72-b941-b77d9b5d7867 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:57.995056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a8e1-aabd-4d72-b941-b77d9b5d7867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:57.995542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a8e1-aabd-4d72-b941-b77d9b5d7867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:57.998259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640a8e1-aabd-4d72-b941-b77d9b5d7867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:58.005138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640a8e1-aabd-4d72-b941-b77d9b5d7867 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:58.036518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 828/1643] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:33:58.040279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4b8a4-f44d-4bce-9e9d-70f83680cb69 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:58.051145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4b8a4-f44d-4bce-9e9d-70f83680cb69 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:58.066577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0e7cd-6f70-48ff-8563-0be4ec6c0b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:58.068218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0e7cd-6f70-48ff-8563-0be4ec6c0b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:58.068784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0e7cd-6f70-48ff-8563-0be4ec6c0b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:58.068994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0e7cd-6f70-48ff-8563-0be4ec6c0b16 admin admin] PATH_INFO: `/v3/projects/f8520fd77c304eab9377377f7bfdf266/users/d2f3043aa48448d0b9cb355b46796f69/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:58.110380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0e7cd-6f70-48ff-8563-0be4ec6c0b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8520fd77c304eab9377377f7bfdf266, user_id=d2f3043aa48448d0b9cb355b46796f69, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:58.114556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0e7cd-6f70-48ff-8563-0be4ec6c0b16 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:58.167712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 816/1644] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:58 2022] PUT /identity/v3/projects/f8520fd77c304eab9377377f7bfdf266/users/d2f3043aa48448d0b9cb355b46796f69/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:58.168727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640a8e1-aabd-4d72-b941-b77d9b5d7867 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:58.187335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640a8e1-aabd-4d72-b941-b77d9b5d7867 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:58.291986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0e7cd-6f70-48ff-8563-0be4ec6c0b16 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:58.294627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-14a0e7cd-6f70-48ff-8563-0be4ec6c0b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:58.342985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cf55-eafd-4de8-9fa9-49ca01eb5f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:58.344316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cf55-eafd-4de8-9fa9-49ca01eb5f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:58.344847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cf55-eafd-4de8-9fa9-49ca01eb5f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:58.345330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cf55-eafd-4de8-9fa9-49ca01eb5f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:58.347085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002cf55-eafd-4de8-9fa9-49ca01eb5f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:58.350439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002cf55-eafd-4de8-9fa9-49ca01eb5f05 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:58.373152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 829/1645] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:33:58.377347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cf55-eafd-4de8-9fa9-49ca01eb5f05 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:58.393015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cf55-eafd-4de8-9fa9-49ca01eb5f05 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:58.406547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce34731-3e6f-42d7-b363-4330144f9969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:58.408327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce34731-3e6f-42d7-b363-4330144f9969 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:58.408770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce34731-3e6f-42d7-b363-4330144f9969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:58.409160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce34731-3e6f-42d7-b363-4330144f9969 admin admin] PATH_INFO: `/v3/projects/f8520fd77c304eab9377377f7bfdf266/users/d2f3043aa48448d0b9cb355b46796f69/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:58.411781 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce34731-3e6f-42d7-b363-4330144f9969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8520fd77c304eab9377377f7bfdf266, user_id=d2f3043aa48448d0b9cb355b46796f69, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:58.414803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce34731-3e6f-42d7-b363-4330144f9969 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:58.432831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 830/1646] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:33:58 2022] PUT /identity/v3/projects/f8520fd77c304eab9377377f7bfdf266/users/d2f3043aa48448d0b9cb355b46796f69/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:58.434724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:58.508058 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0e7cd-6f70-48ff-8563-0be4ec6c0b16 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:58.686990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:58.692468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:58.693183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:58.693480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] PATH_INFO: `/v3/projects/10bcea77b2814563943c60926660f4f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:58.696187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10bcea77b2814563943c60926660f4f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:58.701041 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:33:58.701041 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 21 17:33:58.701041 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: Jul 21 17:33:58.703127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:58.763787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce34731-3e6f-42d7-b363-4330144f9969 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:58.766177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cce34731-3e6f-42d7-b363-4330144f9969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:58.801391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce34731-3e6f-42d7-b363-4330144f9969 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:58.840450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10bcea77b2814563943c60926660f4f8', 'request_id': 'req-ab809118-e6d7-4be8-bdc6-a420f2db5b39'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:58.857924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10bcea77b2814563943c60926660f4f8', 'request_id': 'req-ab809118-e6d7-4be8-bdc6-a420f2db5b39'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:58.869245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10bcea77b2814563943c60926660f4f8', 'request_id': 'req-ab809118-e6d7-4be8-bdc6-a420f2db5b39'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:58.924186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] The token cache is being invalidate because project 10bcea77b2814563943c60926660f4f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:33:58.924470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:58.927228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:58.932778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 817/1647] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:33:58 2022] DELETE /identity/v3/projects/10bcea77b2814563943c60926660f4f8 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:58.937016 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:58.939174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:58.994888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:58.996913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:58.997421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:58.997950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] PATH_INFO: `/v3/users/e940d6ceaa6d48be8dbfe4c6ce6f6238` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:58.999838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e940d6ceaa6d48be8dbfe4c6ce6f6238)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:59.001084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 17:33:59.001084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 21 17:33:59.001084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: Jul 21 17:33:59.003701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:59.109060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab809118-e6d7-4be8-bdc6-a420f2db5b39 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:59.120239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:59.121015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:59.123152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:59.123585 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:59.123868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] PATH_INFO: `/v3/users/2abcd1bfb9834cc3b5873933cc35da73` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:59.129775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2abcd1bfb9834cc3b5873933cc35da73)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:59.131801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:59.163194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:33:59.227949 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e940d6ceaa6d48be8dbfe4c6ce6f6238'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:59.239849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2abcd1bfb9834cc3b5873933cc35da73'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:59.242548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2abcd1bfb9834cc3b5873933cc35da73'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:59.244459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2abcd1bfb9834cc3b5873933cc35da73'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:59.250408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e940d6ceaa6d48be8dbfe4c6ce6f6238'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:59.252110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e940d6ceaa6d48be8dbfe4c6ce6f6238'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:59.260169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2abcd1bfb9834cc3b5873933cc35da73'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:59.292331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bde3c79e-d387-4cce-bef6-04b1cf665687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2abcd1bfb9834cc3b5873933cc35da73'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:59.308271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e940d6ceaa6d48be8dbfe4c6ce6f6238'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:59.319750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 818/1648] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:33:58 2022] DELETE /identity/v3/users/2abcd1bfb9834cc3b5873933cc35da73 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:59.329003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e940d6ceaa6d48be8dbfe4c6ce6f6238'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:33:59.343679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 831/1649] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:33:58 2022] DELETE /identity/v3/users/e940d6ceaa6d48be8dbfe4c6ce6f6238 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:33:59.348590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cc04d-4af7-47a6-b3bf-0c8fe32cb9c7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:59.349242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cc04d-4af7-47a6-b3bf-0c8fe32cb9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:59.349242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cc04d-4af7-47a6-b3bf-0c8fe32cb9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:59.382155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-976cc04d-4af7-47a6-b3bf-0c8fe32cb9c7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:33:59.385960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-976cc04d-4af7-47a6-b3bf-0c8fe32cb9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:59.509860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 819/1650] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:33:59.545867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:33:59.548129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:33:59.718547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e1f08-c659-4ab9-9e2a-8f407c82e193 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:33:59.732043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594db4f7-1cd3-4e47-b2f7-67f780b7fe45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:33:59.734290 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594db4f7-1cd3-4e47-b2f7-67f780b7fe45 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:33:59.734290 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594db4f7-1cd3-4e47-b2f7-67f780b7fe45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:33:59.734542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594db4f7-1cd3-4e47-b2f7-67f780b7fe45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:33:59.764606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594db4f7-1cd3-4e47-b2f7-67f780b7fe45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:33:59.764606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594db4f7-1cd3-4e47-b2f7-67f780b7fe45 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:33:59.786663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 832/1651] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:33:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:01.221146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976cc04d-4af7-47a6-b3bf-0c8fe32cb9c7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:01.222698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-976cc04d-4af7-47a6-b3bf-0c8fe32cb9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:01.254445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976cc04d-4af7-47a6-b3bf-0c8fe32cb9c7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:01.269136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:01.270033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:01.270309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:01.270583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] PATH_INFO: `/v3/projects/5ad49eae079d491f8b27129553c0f628` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:01.274638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad49eae079d491f8b27129553c0f628)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:01.276829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:01.411660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad49eae079d491f8b27129553c0f628', 'request_id': 'req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:01.427271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad49eae079d491f8b27129553c0f628', 'request_id': 'req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:01.442959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad49eae079d491f8b27129553c0f628', 'request_id': 'req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:01.503728 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] The token cache is being invalidate because project 5ad49eae079d491f8b27129553c0f628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:01.504084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:01.505313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:01.510784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 820/1652] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:01 2022] DELETE /identity/v3/projects/5ad49eae079d491f8b27129553c0f628 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:01.518418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80a775-3257-4e23-b4dd-af07cd71ac16 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:01.518868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80a775-3257-4e23-b4dd-af07cd71ac16 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:01.519126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80a775-3257-4e23-b4dd-af07cd71ac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:01.558343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6b80a775-3257-4e23-b4dd-af07cd71ac16 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:01.567387 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6b80a775-3257-4e23-b4dd-af07cd71ac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:01.671736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:01.673432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:01.710303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:01.815290 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 833/1653] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:01.828286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb20a7-b5ba-4ee9-bfdc-0ea01c0ad771 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:01.828834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb20a7-b5ba-4ee9-bfdc-0ea01c0ad771 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:01.829308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb20a7-b5ba-4ee9-bfdc-0ea01c0ad771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:01.874446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4b1ea-844b-45ae-bc95-cb6d4e55278e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:01.914348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:01.915450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:01.917131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:01.919984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] PATH_INFO: `/v3/projects/1fd3b826ab3d4dfe977c098f0071762d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:01.922099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd3b826ab3d4dfe977c098f0071762d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:01.937192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:01.957983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6deb20a7-b5ba-4ee9-bfdc-0ea01c0ad771 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:01.961721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6deb20a7-b5ba-4ee9-bfdc-0ea01c0ad771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:01.985954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 834/1654] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:01.991419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb20a7-b5ba-4ee9-bfdc-0ea01c0ad771 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:02.014034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb20a7-b5ba-4ee9-bfdc-0ea01c0ad771 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:02.055430 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd3b826ab3d4dfe977c098f0071762d', 'request_id': 'req-93e751fa-e4c8-46fe-aacd-05685134ee99'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:02.080175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd3b826ab3d4dfe977c098f0071762d', 'request_id': 'req-93e751fa-e4c8-46fe-aacd-05685134ee99'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:02.088024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd3b826ab3d4dfe977c098f0071762d', 'request_id': 'req-93e751fa-e4c8-46fe-aacd-05685134ee99'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:02.116665 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] The token cache is being invalidate because project 1fd3b826ab3d4dfe977c098f0071762d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:02.117437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:02.117920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-93e751fa-e4c8-46fe-aacd-05685134ee99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:02.119971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 821/1655] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:01 2022] DELETE /identity/v3/projects/1fd3b826ab3d4dfe977c098f0071762d => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:02.125249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:02.126035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207f586-584e-43fd-a43a-e85294c1c0bd None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.126145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207f586-584e-43fd-a43a-e85294c1c0bd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.126401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207f586-584e-43fd-a43a-e85294c1c0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.189185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb82d28-b643-41b9-8487-ffcb85f7eb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:02.189921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb82d28-b643-41b9-8487-ffcb85f7eb3f admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.190209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb82d28-b643-41b9-8487-ffcb85f7eb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.190489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb82d28-b643-41b9-8487-ffcb85f7eb3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.191241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb82d28-b643-41b9-8487-ffcb85f7eb3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:02.191555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb82d28-b643-41b9-8487-ffcb85f7eb3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:02.193686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb82d28-b643-41b9-8487-ffcb85f7eb3f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:02.209356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0207f586-584e-43fd-a43a-e85294c1c0bd None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:02.210817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0207f586-584e-43fd-a43a-e85294c1c0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:02.221021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 835/1656] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:02.222040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb82d28-b643-41b9-8487-ffcb85f7eb3f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:02.224285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3fb82d28-b643-41b9-8487-ffcb85f7eb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:02.238830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 822/1657] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:02.249240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924af74d-4214-4e0f-b7c1-948231231b08 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.249850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924af74d-4214-4e0f-b7c1-948231231b08 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.251597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924af74d-4214-4e0f-b7c1-948231231b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.258679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb82d28-b643-41b9-8487-ffcb85f7eb3f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:02.266779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2b90c-61e6-4b56-b050-e8fe1f830a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:02.267630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b90c-61e6-4b56-b050-e8fe1f830a98 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.268396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b90c-61e6-4b56-b050-e8fe1f830a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.268737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b90c-61e6-4b56-b050-e8fe1f830a98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.269512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2b90c-61e6-4b56-b050-e8fe1f830a98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:02.271636 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2b90c-61e6-4b56-b050-e8fe1f830a98 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:02.301573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-924af74d-4214-4e0f-b7c1-948231231b08 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:02.302650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-924af74d-4214-4e0f-b7c1-948231231b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:02.311459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 836/1658] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:02 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:02.313919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2b90c-61e6-4b56-b050-e8fe1f830a98 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:02.412786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2b90c-61e6-4b56-b050-e8fe1f830a98 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:02.418471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935f1cb-27db-43ac-a22d-e6a257c8d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:02.419765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935f1cb-27db-43ac-a22d-e6a257c8d57d admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.420061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935f1cb-27db-43ac-a22d-e6a257c8d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.420446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935f1cb-27db-43ac-a22d-e6a257c8d57d admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.421525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935f1cb-27db-43ac-a22d-e6a257c8d57d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:02.423688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935f1cb-27db-43ac-a22d-e6a257c8d57d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:02.455207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 837/1659] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:02 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:34:02.458954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935f1cb-27db-43ac-a22d-e6a257c8d57d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:02.471362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935f1cb-27db-43ac-a22d-e6a257c8d57d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:02.477146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aabeed-5aba-4379-a50d-b97f804d3d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:02.478081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aabeed-5aba-4379-a50d-b97f804d3d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.478611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aabeed-5aba-4379-a50d-b97f804d3d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.479075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aabeed-5aba-4379-a50d-b97f804d3d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.480088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aabeed-5aba-4379-a50d-b97f804d3d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:02.484909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aabeed-5aba-4379-a50d-b97f804d3d98 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:02.504374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 838/1660] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:34:02.508435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aabeed-5aba-4379-a50d-b97f804d3d98 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:02.526387 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aabeed-5aba-4379-a50d-b97f804d3d98 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:02.540491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b7e12-5115-4c34-a48b-736782862bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:02.541586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b7e12-5115-4c34-a48b-736782862bae admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.542443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b7e12-5115-4c34-a48b-736782862bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.543463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b7e12-5115-4c34-a48b-736782862bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.545764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b7e12-5115-4c34-a48b-736782862bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:02.548666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b7e12-5115-4c34-a48b-736782862bae admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:02.569690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 839/1661] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:02.574935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b7e12-5115-4c34-a48b-736782862bae admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:02.590255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b7e12-5115-4c34-a48b-736782862bae admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:02.598780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38722d-f374-4f7d-a03b-ed5328f5a072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:02.599801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 823/1662] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:02.603833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38722d-f374-4f7d-a03b-ed5328f5a072 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.604230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38722d-f374-4f7d-a03b-ed5328f5a072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.604938 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924af74d-4214-4e0f-b7c1-948231231b08 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:02.605714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38722d-f374-4f7d-a03b-ed5328f5a072 admin admin] PATH_INFO: `/v3/projects/86fb94b17b6c4c009c0645a2a8fad997/users/af7d6fbf0a3843b0aa8560b4cdf57310/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.637914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924af74d-4214-4e0f-b7c1-948231231b08 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:02.645119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38722d-f374-4f7d-a03b-ed5328f5a072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86fb94b17b6c4c009c0645a2a8fad997, user_id=af7d6fbf0a3843b0aa8560b4cdf57310, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:02.651348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38722d-f374-4f7d-a03b-ed5328f5a072 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:02.671257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff1b5-481e-4e56-82e4-432f0f59b8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:02.671985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff1b5-481e-4e56-82e4-432f0f59b8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.672368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff1b5-481e-4e56-82e4-432f0f59b8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.672612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bff1b5-481e-4e56-82e4-432f0f59b8a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.673426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bff1b5-481e-4e56-82e4-432f0f59b8a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:02.678195 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bff1b5-481e-4e56-82e4-432f0f59b8a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:02.688002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 840/1663] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:02 2022] PUT /identity/v3/projects/86fb94b17b6c4c009c0645a2a8fad997/users/af7d6fbf0a3843b0aa8560b4cdf57310/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:02.690858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bff1b5-481e-4e56-82e4-432f0f59b8a8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:02.694245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c90562-108d-4656-ba5d-1a75684add47 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.694676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c90562-108d-4656-ba5d-1a75684add47 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.695563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c90562-108d-4656-ba5d-1a75684add47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.724378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 824/1664] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:02.728375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff1b5-481e-4e56-82e4-432f0f59b8a8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:02.749521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bff1b5-481e-4e56-82e4-432f0f59b8a8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:02.789380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-68c90562-108d-4656-ba5d-1a75684add47 None None] MFA Rules not processed for user `af7d6fbf0a3843b0aa8560b4cdf57310`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:02.790818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-68c90562-108d-4656-ba5d-1a75684add47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:02.804270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:02.879454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:02.892579 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad17fde-528e-48f2-9e47-27a033e7b998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:02.893469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad17fde-528e-48f2-9e47-27a033e7b998 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:02.893862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad17fde-528e-48f2-9e47-27a033e7b998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:02.894173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad17fde-528e-48f2-9e47-27a033e7b998 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:02.894908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad17fde-528e-48f2-9e47-27a033e7b998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:02.898801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad17fde-528e-48f2-9e47-27a033e7b998 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:02.961350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 825/1665] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:02 2022] POST /identity/v3/projects => generated 367 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:02.965241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad17fde-528e-48f2-9e47-27a033e7b998 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:02.969220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 841/1666] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:02.979580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c90562-108d-4656-ba5d-1a75684add47 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:03.008117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c90562-108d-4656-ba5d-1a75684add47 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:03.112432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad17fde-528e-48f2-9e47-27a033e7b998 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:03.126706 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c9764-1254-4735-b5a6-4584266b35f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:03.127497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c9764-1254-4735-b5a6-4584266b35f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:03.130026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c9764-1254-4735-b5a6-4584266b35f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:03.130270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c9764-1254-4735-b5a6-4584266b35f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:03.131002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c9764-1254-4735-b5a6-4584266b35f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:03.134537 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c9764-1254-4735-b5a6-4584266b35f3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:03.160884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefff86e-3496-4bd4-a4bd-7a69e8793b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:03.161790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefff86e-3496-4bd4-a4bd-7a69e8793b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:03.162175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefff86e-3496-4bd4-a4bd-7a69e8793b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:03.162576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefff86e-3496-4bd4-a4bd-7a69e8793b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:03.163430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefff86e-3496-4bd4-a4bd-7a69e8793b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:03.166461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefff86e-3496-4bd4-a4bd-7a69e8793b6b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:03.172747 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 826/1667] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:02 2022] POST /identity/v3/users => generated 284 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:03.176030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c9764-1254-4735-b5a6-4584266b35f3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:03.191210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c9764-1254-4735-b5a6-4584266b35f3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:03.197966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc056c-db09-4cde-874f-9dcff0952b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:03.199307 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc056c-db09-4cde-874f-9dcff0952b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:03.199706 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc056c-db09-4cde-874f-9dcff0952b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:03.200097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc056c-db09-4cde-874f-9dcff0952b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:03.201003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc056c-db09-4cde-874f-9dcff0952b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:03.202804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc056c-db09-4cde-874f-9dcff0952b6f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:03.204720 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 842/1668] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:02 2022] POST /identity/v3/projects => generated 375 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:03.207612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefff86e-3496-4bd4-a4bd-7a69e8793b6b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:03.224762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 827/1669] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:03.234143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc056c-db09-4cde-874f-9dcff0952b6f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:03.255920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc056c-db09-4cde-874f-9dcff0952b6f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:03.405215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefff86e-3496-4bd4-a4bd-7a69e8793b6b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:03.416794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ec3a3-a6b4-453b-ab86-bbbae26dbee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:03.417667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ec3a3-a6b4-453b-ab86-bbbae26dbee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:03.417787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ec3a3-a6b4-453b-ab86-bbbae26dbee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:03.418657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ec3a3-a6b4-453b-ab86-bbbae26dbee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:03.420487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ec3a3-a6b4-453b-ab86-bbbae26dbee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:03.437615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ec3a3-a6b4-453b-ab86-bbbae26dbee9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:03.453912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150bb38-df7b-4f74-88c9-a4987e2176c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:03.454625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150bb38-df7b-4f74-88c9-a4987e2176c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:03.455097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150bb38-df7b-4f74-88c9-a4987e2176c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:03.455382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150bb38-df7b-4f74-88c9-a4987e2176c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:03.456170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150bb38-df7b-4f74-88c9-a4987e2176c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:03.463679 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150bb38-df7b-4f74-88c9-a4987e2176c6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:03.493541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 843/1670] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:03 2022] POST /identity/v3/users => generated 294 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:03.501545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe1606-a04f-48e1-b328-187499286318 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:03.509619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe1606-a04f-48e1-b328-187499286318 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:03.515276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe1606-a04f-48e1-b328-187499286318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:03.517900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 828/1671] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:03.518939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150bb38-df7b-4f74-88c9-a4987e2176c6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:03.573967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150bb38-df7b-4f74-88c9-a4987e2176c6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:03.592024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10899463-49ab-4141-9b7b-e2333e4a156c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:03.596647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10899463-49ab-4141-9b7b-e2333e4a156c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:03.597019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10899463-49ab-4141-9b7b-e2333e4a156c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:03.598941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10899463-49ab-4141-9b7b-e2333e4a156c admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:03.599740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10899463-49ab-4141-9b7b-e2333e4a156c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:03.601445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10899463-49ab-4141-9b7b-e2333e4a156c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:03.608886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-83fe1606-a04f-48e1-b328-187499286318 None None] MFA Rules not processed for user `d2f3043aa48448d0b9cb355b46796f69`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:03.610125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-83fe1606-a04f-48e1-b328-187499286318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:03.643841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 829/1672] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:03.647877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10899463-49ab-4141-9b7b-e2333e4a156c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:03.675959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10899463-49ab-4141-9b7b-e2333e4a156c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:03.689080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41488080-1ab5-4b79-8528-3d1ce7b386bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:03.690104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41488080-1ab5-4b79-8528-3d1ce7b386bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:03.691823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41488080-1ab5-4b79-8528-3d1ce7b386bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:03.691823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41488080-1ab5-4b79-8528-3d1ce7b386bf admin admin] PATH_INFO: `/v3/projects/5230f46303ec471bb9c78e0feaee0fc1/users/359c8f6d7feb41b69685f85a91411d55/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:03.712893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:03.716398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:03.729175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41488080-1ab5-4b79-8528-3d1ce7b386bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5230f46303ec471bb9c78e0feaee0fc1, user_id=359c8f6d7feb41b69685f85a91411d55, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:03.735733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41488080-1ab5-4b79-8528-3d1ce7b386bf admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:03.763644 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 830/1673] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:03 2022] PUT /identity/v3/projects/5230f46303ec471bb9c78e0feaee0fc1/users/359c8f6d7feb41b69685f85a91411d55/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:03.764261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41488080-1ab5-4b79-8528-3d1ce7b386bf admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:03.802110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 844/1674] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:03.802110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af474ff-b5e5-4a1d-82e6-2f027ca55c29 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:03.802110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af474ff-b5e5-4a1d-82e6-2f027ca55c29 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:03.802110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af474ff-b5e5-4a1d-82e6-2f027ca55c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:03.875556 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5af474ff-b5e5-4a1d-82e6-2f027ca55c29 None None] MFA Rules not processed for user `359c8f6d7feb41b69685f85a91411d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:03.879606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5af474ff-b5e5-4a1d-82e6-2f027ca55c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:03.959026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41488080-1ab5-4b79-8528-3d1ce7b386bf admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:03.979962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27274be8-c058-42a0-ae0d-54a0512989e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:03.982742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27274be8-c058-42a0-ae0d-54a0512989e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:03.983156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27274be8-c058-42a0-ae0d-54a0512989e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:03.983539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27274be8-c058-42a0-ae0d-54a0512989e6 admin admin] PATH_INFO: `/v3/projects/972624793e4b49d8a6c04f1d019649d7/users/bb0124b7fe2d439083fa35b21162a5cb/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:04.018606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27274be8-c058-42a0-ae0d-54a0512989e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=972624793e4b49d8a6c04f1d019649d7, user_id=bb0124b7fe2d439083fa35b21162a5cb, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:04.021885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27274be8-c058-42a0-ae0d-54a0512989e6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:04.052336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 831/1675] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:03 2022] PUT /identity/v3/projects/972624793e4b49d8a6c04f1d019649d7/users/bb0124b7fe2d439083fa35b21162a5cb/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:04.057336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27274be8-c058-42a0-ae0d-54a0512989e6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:04.063282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 845/1676] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:04.067834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af474ff-b5e5-4a1d-82e6-2f027ca55c29 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:04.092211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af474ff-b5e5-4a1d-82e6-2f027ca55c29 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:04.189069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27274be8-c058-42a0-ae0d-54a0512989e6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:04.198048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2711cc-c81e-47e4-ac4d-2aa9c89126e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:04.201097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2711cc-c81e-47e4-ac4d-2aa9c89126e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:04.201713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2711cc-c81e-47e4-ac4d-2aa9c89126e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:04.202808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2711cc-c81e-47e4-ac4d-2aa9c89126e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:04.204244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2711cc-c81e-47e4-ac4d-2aa9c89126e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:04.207706 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2711cc-c81e-47e4-ac4d-2aa9c89126e9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:04.232022 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 832/1677] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:04.233842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2711cc-c81e-47e4-ac4d-2aa9c89126e9 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:04.243881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b3d4c-8074-49f6-9f78-5b49f8f3f4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:04.244588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b3d4c-8074-49f6-9f78-5b49f8f3f4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:04.244876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b3d4c-8074-49f6-9f78-5b49f8f3f4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:04.246156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b3d4c-8074-49f6-9f78-5b49f8f3f4b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:04.247987 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b3d4c-8074-49f6-9f78-5b49f8f3f4b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:04.249768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b3d4c-8074-49f6-9f78-5b49f8f3f4b8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:04.253671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2711cc-c81e-47e4-ac4d-2aa9c89126e9 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:04.265662 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34bad0-822d-4573-9da8-160b97bf507e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:04.267661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34bad0-822d-4573-9da8-160b97bf507e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:04.268138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34bad0-822d-4573-9da8-160b97bf507e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:04.269352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34bad0-822d-4573-9da8-160b97bf507e admin admin] PATH_INFO: `/v3/projects/972624793e4b49d8a6c04f1d019649d7/users/bb0124b7fe2d439083fa35b21162a5cb/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:04.277858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34bad0-822d-4573-9da8-160b97bf507e admin admin] RBAC: Authorizing `identity:create_grant(project_id=972624793e4b49d8a6c04f1d019649d7, user_id=bb0124b7fe2d439083fa35b21162a5cb, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:04.281571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34bad0-822d-4573-9da8-160b97bf507e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:04.314717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 846/1679] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:04 2022] POST /identity/v3/projects => generated 367 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:34:04.323962 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91ce97-17b0-4d7e-bd73-ff497e42f7a1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:04.324813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91ce97-17b0-4d7e-bd73-ff497e42f7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:04.325095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91ce97-17b0-4d7e-bd73-ff497e42f7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:04.325668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 833/1679] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:04 2022] PUT /identity/v3/projects/972624793e4b49d8a6c04f1d019649d7/users/bb0124b7fe2d439083fa35b21162a5cb/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:04.333584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34bad0-822d-4573-9da8-160b97bf507e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:04.395620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-aa91ce97-17b0-4d7e-bd73-ff497e42f7a1 None None] MFA Rules not processed for user `bb0124b7fe2d439083fa35b21162a5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:04.396873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-aa91ce97-17b0-4d7e-bd73-ff497e42f7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:04.439259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:04.451432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34bad0-822d-4573-9da8-160b97bf507e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:04.529525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:04.568845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55719d-6857-4fc2-996e-38cc2db3f201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:04.569461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55719d-6857-4fc2-996e-38cc2db3f201 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:04.569813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55719d-6857-4fc2-996e-38cc2db3f201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:04.570157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55719d-6857-4fc2-996e-38cc2db3f201 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:04.570923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55719d-6857-4fc2-996e-38cc2db3f201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:04.573675 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55719d-6857-4fc2-996e-38cc2db3f201 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:04.594079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 847/1680] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:04.595530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a241b-e9a9-4fe6-b9d9-1ba5132225d0 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:04.595903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a241b-e9a9-4fe6-b9d9-1ba5132225d0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:04.596750 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a241b-e9a9-4fe6-b9d9-1ba5132225d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:04.617712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 834/1681] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:04 2022] POST /identity/v3/users => generated 290 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:04.618667 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55719d-6857-4fc2-996e-38cc2db3f201 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:04.621833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fd55719d-6857-4fc2-996e-38cc2db3f201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:04.677666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6e2a241b-e9a9-4fe6-b9d9-1ba5132225d0 None None] MFA Rules not processed for user `fa82bddf9aa0455797d42993fe850ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:04.678592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6e2a241b-e9a9-4fe6-b9d9-1ba5132225d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:04.750491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55719d-6857-4fc2-996e-38cc2db3f201 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:04.764845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9adfa1-eda4-479a-9a49-5173356c183b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:04.765519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9adfa1-eda4-479a-9a49-5173356c183b service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:04.765800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9adfa1-eda4-479a-9a49-5173356c183b service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:04.766074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9adfa1-eda4-479a-9a49-5173356c183b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:04.791842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9adfa1-eda4-479a-9a49-5173356c183b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:04.794560 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9adfa1-eda4-479a-9a49-5173356c183b service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:04.819986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 848/1682] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:04.819986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a241b-e9a9-4fe6-b9d9-1ba5132225d0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:04.835451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 835/1683] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:04.836476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9adfa1-eda4-479a-9a49-5173356c183b service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:04.844078 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a241b-e9a9-4fe6-b9d9-1ba5132225d0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:04.844959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9c9adfa1-eda4-479a-9a49-5173356c183b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:04.854580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169bbb-2665-4cf2-adb2-87e1963fe2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:04.855901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169bbb-2665-4cf2-adb2-87e1963fe2ee admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:04.856195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169bbb-2665-4cf2-adb2-87e1963fe2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:04.856528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169bbb-2665-4cf2-adb2-87e1963fe2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:04.857285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc169bbb-2665-4cf2-adb2-87e1963fe2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:04.861370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc169bbb-2665-4cf2-adb2-87e1963fe2ee admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:04.882686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 849/1684] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:04.890379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169bbb-2665-4cf2-adb2-87e1963fe2ee admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:04.907673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169bbb-2665-4cf2-adb2-87e1963fe2ee admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:04.917446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2cfa3-b336-4804-8195-9ee109c92c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:04.917446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2cfa3-b336-4804-8195-9ee109c92c34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:04.917446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2cfa3-b336-4804-8195-9ee109c92c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:04.917446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2cfa3-b336-4804-8195-9ee109c92c34 admin admin] PATH_INFO: `/v3/projects/f62992d517c74e68a8528e32345eab77/users/760312586f42485ebe51b0c7130b4904/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:04.984807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2cfa3-b336-4804-8195-9ee109c92c34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62992d517c74e68a8528e32345eab77, user_id=760312586f42485ebe51b0c7130b4904, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:04.987501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2cfa3-b336-4804-8195-9ee109c92c34 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:05.021472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 850/1685] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:04 2022] PUT /identity/v3/projects/f62992d517c74e68a8528e32345eab77/users/760312586f42485ebe51b0c7130b4904/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:05.026168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2cfa3-b336-4804-8195-9ee109c92c34 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:05.094658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9adfa1-eda4-479a-9a49-5173356c183b service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:05.211735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2cfa3-b336-4804-8195-9ee109c92c34 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:05.213345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:05.218918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0b890-cb29-4fa5-99b5-dd01663c4cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:05.219749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0b890-cb29-4fa5-99b5-dd01663c4cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:05.220104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0b890-cb29-4fa5-99b5-dd01663c4cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:05.220491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0b890-cb29-4fa5-99b5-dd01663c4cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:05.222836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0b890-cb29-4fa5-99b5-dd01663c4cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:05.223661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0b890-cb29-4fa5-99b5-dd01663c4cbe admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:05.248490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 851/1686] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:05.254778 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0b890-cb29-4fa5-99b5-dd01663c4cbe admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:05.276121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0b890-cb29-4fa5-99b5-dd01663c4cbe admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:05.291040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94435d-70cd-4349-a39a-43781fe95aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:05.291953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94435d-70cd-4349-a39a-43781fe95aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:05.292238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94435d-70cd-4349-a39a-43781fe95aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:05.292542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94435d-70cd-4349-a39a-43781fe95aa8 admin admin] PATH_INFO: `/v3/projects/f62992d517c74e68a8528e32345eab77/users/760312586f42485ebe51b0c7130b4904/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:05.298596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94435d-70cd-4349-a39a-43781fe95aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62992d517c74e68a8528e32345eab77, user_id=760312586f42485ebe51b0c7130b4904, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:05.302442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94435d-70cd-4349-a39a-43781fe95aa8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:05.334533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7db16-f358-4de2-8db1-4c558fc29c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:05.335902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7db16-f358-4de2-8db1-4c558fc29c1c service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:05.336194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7db16-f358-4de2-8db1-4c558fc29c1c service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:05.337159 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7db16-f358-4de2-8db1-4c558fc29c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:05.342594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 852/1687] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:05 2022] PUT /identity/v3/projects/f62992d517c74e68a8528e32345eab77/users/760312586f42485ebe51b0c7130b4904/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:05.357218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bc890-2f7a-4ea6-b962-ca0450c5e26e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:05.357571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bc890-2f7a-4ea6-b962-ca0450c5e26e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:05.357881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bc890-2f7a-4ea6-b962-ca0450c5e26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:05.431085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d21bc890-2f7a-4ea6-b962-ca0450c5e26e None None] MFA Rules not processed for user `760312586f42485ebe51b0c7130b4904`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:05.432264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d21bc890-2f7a-4ea6-b962-ca0450c5e26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:05.445088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:05.523449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7db16-f358-4de2-8db1-4c558fc29c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:05.528889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7db16-f358-4de2-8db1-4c558fc29c1c service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:05.555200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 836/1688] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:05.648315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7db16-f358-4de2-8db1-4c558fc29c1c service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:05.649304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 853/1689] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:05.662466 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bc890-2f7a-4ea6-b962-ca0450c5e26e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:05.664080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d21bc890-2f7a-4ea6-b962-ca0450c5e26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:05.735006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7db16-f358-4de2-8db1-4c558fc29c1c service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:05.813512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bc890-2f7a-4ea6-b962-ca0450c5e26e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:05.818310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95efb3-f7e7-41eb-83c9-d95aafe6ea65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:05.820439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95efb3-f7e7-41eb-83c9-d95aafe6ea65 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:05.821366 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95efb3-f7e7-41eb-83c9-d95aafe6ea65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:05.823892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95efb3-f7e7-41eb-83c9-d95aafe6ea65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:05.838902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95efb3-f7e7-41eb-83c9-d95aafe6ea65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:05.842915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95efb3-f7e7-41eb-83c9-d95aafe6ea65 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:05.862081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 854/1690] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:34:05.902270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c1697-e910-47fd-869b-43dca652a8e7 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0124b7fe2d439083fa35b21162a5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '972624793e4b49d8a6c04f1d019649d7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:05.903410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1697-e910-47fd-869b-43dca652a8e7 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:05.903696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1697-e910-47fd-869b-43dca652a8e7 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:05.903979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1697-e910-47fd-869b-43dca652a8e7 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:05.905061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c1697-e910-47fd-869b-43dca652a8e7 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:05.905061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c1697-e910-47fd-869b-43dca652a8e7 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:05.906678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c1697-e910-47fd-869b-43dca652a8e7 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:05.951809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 837/1691] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:05.953535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95efb3-f7e7-41eb-83c9-d95aafe6ea65 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:05.975385 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95efb3-f7e7-41eb-83c9-d95aafe6ea65 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:05.987614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594274c8-d5cd-481c-b3e4-5eebe40e80a1 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0124b7fe2d439083fa35b21162a5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '972624793e4b49d8a6c04f1d019649d7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:05.988334 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594274c8-d5cd-481c-b3e4-5eebe40e80a1 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:05.988632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594274c8-d5cd-481c-b3e4-5eebe40e80a1 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:05.988902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594274c8-d5cd-481c-b3e4-5eebe40e80a1 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:05.989623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594274c8-d5cd-481c-b3e4-5eebe40e80a1 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:06.001408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594274c8-d5cd-481c-b3e4-5eebe40e80a1 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:06.084905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 855/1692] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:05 2022] POST /identity/v3/projects => generated 354 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:06.088326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c1697-e910-47fd-869b-43dca652a8e7 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:06.115865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c1697-e910-47fd-869b-43dca652a8e7 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:06.170734 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:06.383164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594274c8-d5cd-481c-b3e4-5eebe40e80a1 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:06.389478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-594274c8-d5cd-481c-b3e4-5eebe40e80a1 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:06.403465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4cafe-a800-4dc1-ad03-ebb2d3dafa8f tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0124b7fe2d439083fa35b21162a5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '972624793e4b49d8a6c04f1d019649d7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:06.404789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4cafe-a800-4dc1-ad03-ebb2d3dafa8f tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:06.404789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4cafe-a800-4dc1-ad03-ebb2d3dafa8f tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:06.404789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4cafe-a800-4dc1-ad03-ebb2d3dafa8f tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:06.405464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4cafe-a800-4dc1-ad03-ebb2d3dafa8f tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:06.405782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4cafe-a800-4dc1-ad03-ebb2d3dafa8f tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:06.411746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4cafe-a800-4dc1-ad03-ebb2d3dafa8f tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:06.441225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 838/1693] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:06.527963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4cafe-a800-4dc1-ad03-ebb2d3dafa8f tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:06.543065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4cafe-a800-4dc1-ad03-ebb2d3dafa8f tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:06.571372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0124b7fe2d439083fa35b21162a5cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '972624793e4b49d8a6c04f1d019649d7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:06.575131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:06.586371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:06.586841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] PATH_INFO: `/v3/projects/ec9008f8628044bfbe920a3389429106` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:06.594705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ec9008f8628044bfbe920a3389429106)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:06.597104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:06.610741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594274c8-d5cd-481c-b3e4-5eebe40e80a1 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:06.625123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:06.626099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:06.627765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:06.628068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] PATH_INFO: `/v3/users/0202ce3a9ad3475892f7115bf9ff7567` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:06.629990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0202ce3a9ad3475892f7115bf9ff7567)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:06.634188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:06.700239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:06.719377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec9008f8628044bfbe920a3389429106', 'request_id': 'req-7adc1fe3-98e0-44d4-8a10-150407e3d001'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:06.743682 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec9008f8628044bfbe920a3389429106', 'request_id': 'req-7adc1fe3-98e0-44d4-8a10-150407e3d001'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:06.757944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec9008f8628044bfbe920a3389429106', 'request_id': 'req-7adc1fe3-98e0-44d4-8a10-150407e3d001'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:06.769284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0202ce3a9ad3475892f7115bf9ff7567'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:06.784264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0202ce3a9ad3475892f7115bf9ff7567'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:06.785609 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0202ce3a9ad3475892f7115bf9ff7567'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:06.822322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] The token cache is being invalidate because project ec9008f8628044bfbe920a3389429106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:06.822614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:06.827881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0202ce3a9ad3475892f7115bf9ff7567'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:06.828667 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7adc1fe3-98e0-44d4-8a10-150407e3d001 tempest-VolumeQuotaClassesTest-7705578 tempest-VolumeQuotaClassesTest-7705578-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:06.841816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 839/1694] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:06 2022] DELETE /identity/v3/projects/ec9008f8628044bfbe920a3389429106 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:06.887303 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0202ce3a9ad3475892f7115bf9ff7567'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:06.909888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 856/1695] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:06 2022] DELETE /identity/v3/users/0202ce3a9ad3475892f7115bf9ff7567 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:06.922126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f901-d67a-421a-923c-dcde5e65fb0e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:06.922332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f901-d67a-421a-923c-dcde5e65fb0e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:06.922609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f901-d67a-421a-923c-dcde5e65fb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:06.954464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ef23f901-d67a-421a-923c-dcde5e65fb0e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:06.956099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ef23f901-d67a-421a-923c-dcde5e65fb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:07.127837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 840/1696] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:07.294076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:07.296151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:07.323218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f267d-c714-46c9-b101-fed7ef5f64fc admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:07.339289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:07.341061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:07.341252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:07.341502 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] PATH_INFO: `/v3/users/af7d6fbf0a3843b0aa8560b4cdf57310` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:07.347073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af7d6fbf0a3843b0aa8560b4cdf57310)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:07.349147 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23f901-d67a-421a-923c-dcde5e65fb0e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:07.349531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:07.350854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ef23f901-d67a-421a-923c-dcde5e65fb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:07.442258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:07.497684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af7d6fbf0a3843b0aa8560b4cdf57310'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:07.511931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af7d6fbf0a3843b0aa8560b4cdf57310'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:07.514044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af7d6fbf0a3843b0aa8560b4cdf57310'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:07.517122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23f901-d67a-421a-923c-dcde5e65fb0e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:07.525199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0c3ee-10fd-444f-a9cf-5dd3a56f7a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:07.525878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0c3ee-10fd-444f-a9cf-5dd3a56f7a76 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:07.526166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0c3ee-10fd-444f-a9cf-5dd3a56f7a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:07.526440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0c3ee-10fd-444f-a9cf-5dd3a56f7a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:07.555454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af7d6fbf0a3843b0aa8560b4cdf57310'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:07.557717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0c3ee-10fd-444f-a9cf-5dd3a56f7a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:07.561384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0c3ee-10fd-444f-a9cf-5dd3a56f7a76 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:07.574443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2e86a367-9a74-43be-94d5-2325ed371039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af7d6fbf0a3843b0aa8560b4cdf57310'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:07.590291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 857/1697] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:07 2022] DELETE /identity/v3/users/af7d6fbf0a3843b0aa8560b4cdf57310 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:07.594702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40350-c45f-4944-a931-3b12a5ebf543 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:07.595804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40350-c45f-4944-a931-3b12a5ebf543 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:07.596938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40350-c45f-4944-a931-3b12a5ebf543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:07.599892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 841/1698] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:07.623448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c9c40350-c45f-4944-a931-3b12a5ebf543 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:07.624720 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c9c40350-c45f-4944-a931-3b12a5ebf543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:07.675299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d28d84-5dd3-4154-b404-d6a43b49932b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:07.675799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d28d84-5dd3-4154-b404-d6a43b49932b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:07.678362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d28d84-5dd3-4154-b404-d6a43b49932b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:07.683944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 842/1699] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 17:34:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:34:07.694819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe704-61a4-428a-baef-7e622ba6dfa6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:07.695145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe704-61a4-428a-baef-7e622ba6dfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:07.695431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe704-61a4-428a-baef-7e622ba6dfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:07.737153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d8dbe704-61a4-428a-baef-7e622ba6dfa6 None None] MFA Rules not processed for user `901cf70064c34b63a0eda979a5ce77e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:07.739782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d8dbe704-61a4-428a-baef-7e622ba6dfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:07.798296 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 858/1700] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:07.888857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 843/1701] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 17:34:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:07.926655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c40350-c45f-4944-a931-3b12a5ebf543 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:07.940638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c9c40350-c45f-4944-a931-3b12a5ebf543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:07.974342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:08.027229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a89b-d601-442c-af64-d71770b4df12 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:08.029370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a89b-d601-442c-af64-d71770b4df12 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:08.029952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a89b-d601-442c-af64-d71770b4df12 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:08.035865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 844/1702] 10.222.0.57 () {54 vars in 961 bytes} [Thu Jul 21 17:34:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 17:34:08.050805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57784b5f-4d49-44c7-912b-bbd10f15fd29 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:08.051209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57784b5f-4d49-44c7-912b-bbd10f15fd29 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:08.055404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57784b5f-4d49-44c7-912b-bbd10f15fd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:08.107148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-57784b5f-4d49-44c7-912b-bbd10f15fd29 None None] MFA Rules not processed for user `901cf70064c34b63a0eda979a5ce77e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:08.110677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-57784b5f-4d49-44c7-912b-bbd10f15fd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:08.146488 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 845/1703] 10.222.0.57 () {58 vars in 1058 bytes} [Thu Jul 21 17:34:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:08.147996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57784b5f-4d49-44c7-912b-bbd10f15fd29 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:08.176742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c40350-c45f-4944-a931-3b12a5ebf543 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:08.176875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57784b5f-4d49-44c7-912b-bbd10f15fd29 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:08.192894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2093b95-0c64-4c15-a65c-b2c0c139933b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:08.196235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2093b95-0c64-4c15-a65c-b2c0c139933b service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:08.196235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2093b95-0c64-4c15-a65c-b2c0c139933b service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:08.196594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2093b95-0c64-4c15-a65c-b2c0c139933b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:08.201876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2eeb8b-f689-4a81-ab4f-96094fe774b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:08.202628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2eeb8b-f689-4a81-ab4f-96094fe774b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:08.203157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2eeb8b-f689-4a81-ab4f-96094fe774b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:08.203508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2eeb8b-f689-4a81-ab4f-96094fe774b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:08.223197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2093b95-0c64-4c15-a65c-b2c0c139933b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:08.227131 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2eeb8b-f689-4a81-ab4f-96094fe774b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:08.229618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2eeb8b-f689-4a81-ab4f-96094fe774b8 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:08.230019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2093b95-0c64-4c15-a65c-b2c0c139933b service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:08.264666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 846/1704] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:08.272189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 859/1705] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:08.805447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2093b95-0c64-4c15-a65c-b2c0c139933b service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:08.807354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b2093b95-0c64-4c15-a65c-b2c0c139933b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:08.835257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2093b95-0c64-4c15-a65c-b2c0c139933b service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:08.852832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:08.853758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:08.854041 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:08.854317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] PATH_INFO: `/v3/users/359c8f6d7feb41b69685f85a91411d55` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:08.861450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=359c8f6d7feb41b69685f85a91411d55)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:08.863427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:09.005273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '359c8f6d7feb41b69685f85a91411d55'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:09.006170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '359c8f6d7feb41b69685f85a91411d55'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:09.006833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '359c8f6d7feb41b69685f85a91411d55'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:09.023943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '359c8f6d7feb41b69685f85a91411d55'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:09.065428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '359c8f6d7feb41b69685f85a91411d55'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:09.096475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 847/1706] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:08 2022] DELETE /identity/v3/users/359c8f6d7feb41b69685f85a91411d55 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:09.100995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eee23-d56b-43f0-b6bc-b52682cc35ac None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:09.101892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eee23-d56b-43f0-b6bc-b52682cc35ac None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:09.102383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eee23-d56b-43f0-b6bc-b52682cc35ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:09.150368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bf0eee23-d56b-43f0-b6bc-b52682cc35ac None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:09.158894 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bf0eee23-d56b-43f0-b6bc-b52682cc35ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:09.348049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 860/1707] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:09.376518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:09.377973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:09.434212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:09.553210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e69d90-bef7-4c20-9b8a-985b2ac702bd admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:09.595467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a7eff-fb0f-48c9-9954-e31db6f8d50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:09.596155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a7eff-fb0f-48c9-9954-e31db6f8d50a service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:09.596497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a7eff-fb0f-48c9-9954-e31db6f8d50a service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:09.596778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a7eff-fb0f-48c9-9954-e31db6f8d50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:09.610105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a7eff-fb0f-48c9-9954-e31db6f8d50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:09.614386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a7eff-fb0f-48c9-9954-e31db6f8d50a service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:09.635825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 848/1708] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:10.421290 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0eee23-d56b-43f0-b6bc-b52682cc35ac None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:10.423269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bf0eee23-d56b-43f0-b6bc-b52682cc35ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:10.447191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0eee23-d56b-43f0-b6bc-b52682cc35ac None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:10.465689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:10.466614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:10.466881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:10.467543 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] PATH_INFO: `/v3/users/83e8fe8d447840dea4510dccf5755a15` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:10.469167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e8fe8d447840dea4510dccf5755a15)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:10.473719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:10.494253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a7eff-fb0f-48c9-9954-e31db6f8d50a service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:10.496049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c60a7eff-fb0f-48c9-9954-e31db6f8d50a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:10.533420 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a7eff-fb0f-48c9-9954-e31db6f8d50a service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:10.561431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:10.562299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:10.562623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:10.562891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] PATH_INFO: `/v3/projects/29e74e3f52ac4b6eb555f458eff85297` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:10.564735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29e74e3f52ac4b6eb555f458eff85297)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:10.570419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:10.640889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e8fe8d447840dea4510dccf5755a15'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:10.659837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e8fe8d447840dea4510dccf5755a15'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:10.660821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e8fe8d447840dea4510dccf5755a15'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:10.707712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e8fe8d447840dea4510dccf5755a15'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:10.711315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29e74e3f52ac4b6eb555f458eff85297', 'request_id': 'req-04e15d26-161f-411c-89cf-1fc4fd342e02'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:10.729526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29e74e3f52ac4b6eb555f458eff85297', 'request_id': 'req-04e15d26-161f-411c-89cf-1fc4fd342e02'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:10.735092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1597e74-deb4-4453-9a90-af85aedd8d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e8fe8d447840dea4510dccf5755a15'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:10.765708 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 861/1709] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:10 2022] DELETE /identity/v3/users/83e8fe8d447840dea4510dccf5755a15 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:10.766026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29e74e3f52ac4b6eb555f458eff85297', 'request_id': 'req-04e15d26-161f-411c-89cf-1fc4fd342e02'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:10.793867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] The token cache is being invalidate because project 29e74e3f52ac4b6eb555f458eff85297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:10.794319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:10.795690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-04e15d26-161f-411c-89cf-1fc4fd342e02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:10.799484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 849/1710] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:10 2022] DELETE /identity/v3/projects/29e74e3f52ac4b6eb555f458eff85297 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:10.804400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f3d48-0c0f-4435-ab2c-579572f9680d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:10.806426 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f3d48-0c0f-4435-ab2c-579572f9680d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:10.806813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f3d48-0c0f-4435-ab2c-579572f9680d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:10.857249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-580f3d48-0c0f-4435-ab2c-579572f9680d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:10.861201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-580f3d48-0c0f-4435-ab2c-579572f9680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:10.940285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:11.012821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 862/1711] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:11.020572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925a024-3f8e-4957-b590-6c25cc2c9a93 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:11.021016 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925a024-3f8e-4957-b590-6c25cc2c9a93 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:11.021394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925a024-3f8e-4957-b590-6c25cc2c9a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:11.049672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3925a024-3f8e-4957-b590-6c25cc2c9a93 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:11.051065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3925a024-3f8e-4957-b590-6c25cc2c9a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:11.061423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 850/1712] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:11.065505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f3d48-0c0f-4435-ab2c-579572f9680d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:11.080113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f3d48-0c0f-4435-ab2c-579572f9680d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:11.092891 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fcbd7-f562-4155-923e-754fda8bd6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:11.093582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fcbd7-f562-4155-923e-754fda8bd6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:11.093864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fcbd7-f562-4155-923e-754fda8bd6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:11.094138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fcbd7-f562-4155-923e-754fda8bd6bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:11.094902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fcbd7-f562-4155-923e-754fda8bd6bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:11.095209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fcbd7-f562-4155-923e-754fda8bd6bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:11.100224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fcbd7-f562-4155-923e-754fda8bd6bd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:11.129047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 863/1713] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:11.133217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925a024-3f8e-4957-b590-6c25cc2c9a93 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:11.151394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925a024-3f8e-4957-b590-6c25cc2c9a93 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:11.166392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0042b-ea4b-457f-b91a-5cce1412617c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:11.167095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0042b-ea4b-457f-b91a-5cce1412617c admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:11.167382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0042b-ea4b-457f-b91a-5cce1412617c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:11.168224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0042b-ea4b-457f-b91a-5cce1412617c admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:11.168982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0042b-ea4b-457f-b91a-5cce1412617c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:11.171099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0042b-ea4b-457f-b91a-5cce1412617c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:11.245024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 851/1714] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:11 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:11.249788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fcbd7-f562-4155-923e-754fda8bd6bd admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:11.293472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fcbd7-f562-4155-923e-754fda8bd6bd admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:11.503167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322f6b8-c077-4b56-8a5d-adcb5b5a18ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:11.505081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322f6b8-c077-4b56-8a5d-adcb5b5a18ef admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:11.505528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322f6b8-c077-4b56-8a5d-adcb5b5a18ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:11.505981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322f6b8-c077-4b56-8a5d-adcb5b5a18ef admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:11.507635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322f6b8-c077-4b56-8a5d-adcb5b5a18ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:11.509829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322f6b8-c077-4b56-8a5d-adcb5b5a18ef admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:11.547315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 864/1715] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:11 2022] POST /identity/v3/users => generated 282 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:11.551035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0042b-ea4b-457f-b91a-5cce1412617c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:11.568005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0042b-ea4b-457f-b91a-5cce1412617c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:11.581408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eef141-0521-49d8-a6ba-57c59fadc1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:11.582382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef141-0521-49d8-a6ba-57c59fadc1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:11.582851 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef141-0521-49d8-a6ba-57c59fadc1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:11.583888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef141-0521-49d8-a6ba-57c59fadc1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:11.586722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eef141-0521-49d8-a6ba-57c59fadc1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:11.593229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eef141-0521-49d8-a6ba-57c59fadc1a5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:11.622570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 852/1716] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:11.624271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322f6b8-c077-4b56-8a5d-adcb5b5a18ef admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:11.645441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322f6b8-c077-4b56-8a5d-adcb5b5a18ef admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:11.657155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b56c3-615a-4beb-9b70-09bf384d0881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:11.658886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b56c3-615a-4beb-9b70-09bf384d0881 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:11.659315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b56c3-615a-4beb-9b70-09bf384d0881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:11.659725 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b56c3-615a-4beb-9b70-09bf384d0881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:11.660709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b56c3-615a-4beb-9b70-09bf384d0881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:11.663730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b56c3-615a-4beb-9b70-09bf384d0881 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:11.706250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 865/1717] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:11.710148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eef141-0521-49d8-a6ba-57c59fadc1a5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:11.731859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eef141-0521-49d8-a6ba-57c59fadc1a5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:11.747043 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd376a-f27c-4893-aa94-8511a572e4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:11.748194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd376a-f27c-4893-aa94-8511a572e4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:11.748531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd376a-f27c-4893-aa94-8511a572e4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:11.748810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd376a-f27c-4893-aa94-8511a572e4c8 admin admin] PATH_INFO: `/v3/projects/e02d044094384d47bb23a2a9580e3b54/users/4fbf0c15ce10455ba894fbb225f25b40/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:11.788617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd376a-f27c-4893-aa94-8511a572e4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02d044094384d47bb23a2a9580e3b54, user_id=4fbf0c15ce10455ba894fbb225f25b40, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:11.798841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd376a-f27c-4893-aa94-8511a572e4c8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:11.805009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b56c3-615a-4beb-9b70-09bf384d0881 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:11.807417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-db9b56c3-615a-4beb-9b70-09bf384d0881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:11.841214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b56c3-615a-4beb-9b70-09bf384d0881 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:11.858228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 853/1718] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:11 2022] PUT /identity/v3/projects/e02d044094384d47bb23a2a9580e3b54/users/4fbf0c15ce10455ba894fbb225f25b40/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:11.869447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64669a5b-e1d0-45a1-a295-8c81ad744442 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:11.869729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64669a5b-e1d0-45a1-a295-8c81ad744442 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:11.870350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64669a5b-e1d0-45a1-a295-8c81ad744442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:11.945903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-64669a5b-e1d0-45a1-a295-8c81ad744442 None None] MFA Rules not processed for user `4fbf0c15ce10455ba894fbb225f25b40`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:11.949564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-64669a5b-e1d0-45a1-a295-8c81ad744442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:11.984210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:11.986037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:11.986537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:11.987087 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] PATH_INFO: `/v3/projects/86fb94b17b6c4c009c0645a2a8fad997` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:11.988792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86fb94b17b6c4c009c0645a2a8fad997)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:11.991128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:12.018607 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:12.063923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:12.083115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86fb94b17b6c4c009c0645a2a8fad997', 'request_id': 'req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.109133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 854/1719] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:11 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:12.109554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86fb94b17b6c4c009c0645a2a8fad997', 'request_id': 'req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.111532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64669a5b-e1d0-45a1-a295-8c81ad744442 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:12.127085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86fb94b17b6c4c009c0645a2a8fad997', 'request_id': 'req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.150207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] The token cache is being invalidate because project 86fb94b17b6c4c009c0645a2a8fad997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:12.150805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.151670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5c6b8a5d-0a3b-4302-99a3-8f58c7d98a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.155727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 866/1720] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:11 2022] DELETE /identity/v3/projects/86fb94b17b6c4c009c0645a2a8fad997 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:12.157689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c6ea4-49db-451d-ae51-43af3dc1a328 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:12.158140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c6ea4-49db-451d-ae51-43af3dc1a328 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:12.158586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c6ea4-49db-451d-ae51-43af3dc1a328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:12.178199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64669a5b-e1d0-45a1-a295-8c81ad744442 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:12.212637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c56c6ea4-49db-451d-ae51-43af3dc1a328 None None] MFA Rules not processed for user `2ea3ea36439143b8a55f181170efc14a`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:12.215377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c56c6ea4-49db-451d-ae51-43af3dc1a328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:12.311608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d1851-f7d7-48a4-a49e-1710df794ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:12.312678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d1851-f7d7-48a4-a49e-1710df794ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:12.312979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d1851-f7d7-48a4-a49e-1710df794ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:12.313342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d1851-f7d7-48a4-a49e-1710df794ad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:12.313944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d1851-f7d7-48a4-a49e-1710df794ad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:12.316636 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 867/1721] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:12.318123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c6ea4-49db-451d-ae51-43af3dc1a328 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:12.320456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c56c6ea4-49db-451d-ae51-43af3dc1a328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:12.322289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d1851-f7d7-48a4-a49e-1710df794ad7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:12.356206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c6ea4-49db-451d-ae51-43af3dc1a328 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:12.362402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:12.363639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:12.364217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:12.364692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] PATH_INFO: `/v3/users/bb0124b7fe2d439083fa35b21162a5cb` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:12.370543 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb0124b7fe2d439083fa35b21162a5cb)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:12.378836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:12.387685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 855/1722] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:12 2022] POST /identity/v3/projects => generated 365 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:12.388352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d1851-f7d7-48a4-a49e-1710df794ad7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:12.390292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-580d1851-f7d7-48a4-a49e-1710df794ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:12.474368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb0124b7fe2d439083fa35b21162a5cb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.485473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb0124b7fe2d439083fa35b21162a5cb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.486786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb0124b7fe2d439083fa35b21162a5cb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.505332 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb0124b7fe2d439083fa35b21162a5cb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.520972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb0124b7fe2d439083fa35b21162a5cb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.530282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d1851-f7d7-48a4-a49e-1710df794ad7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:12.539028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:12.540046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:12.540699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:12.541038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] PATH_INFO: `/v3/projects/e21cd5e8f00a43f4a32eb6390b585b59` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:12.543081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e21cd5e8f00a43f4a32eb6390b585b59)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:12.545294 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:12.549239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 868/1723] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:12 2022] DELETE /identity/v3/users/bb0124b7fe2d439083fa35b21162a5cb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:12.550256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:12.567699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:12.638607 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e21cd5e8f00a43f4a32eb6390b585b59', 'request_id': 'req-800404ea-8046-4894-811a-704b4ca2f5e9'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.658121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e21cd5e8f00a43f4a32eb6390b585b59', 'request_id': 'req-800404ea-8046-4894-811a-704b4ca2f5e9'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.662427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:12.672016 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e21cd5e8f00a43f4a32eb6390b585b59', 'request_id': 'req-800404ea-8046-4894-811a-704b4ca2f5e9'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.681573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:12.712613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] The token cache is being invalidate because project e21cd5e8f00a43f4a32eb6390b585b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:12.713449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.716784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-800404ea-8046-4894-811a-704b4ca2f5e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.727502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 856/1724] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:12 2022] DELETE /identity/v3/projects/e21cd5e8f00a43f4a32eb6390b585b59 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:12.729914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffed09-9b29-4bb0-b781-0a470d878dd4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:12.731254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffed09-9b29-4bb0-b781-0a470d878dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:12.731824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffed09-9b29-4bb0-b781-0a470d878dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:12.762169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d13c2-8309-4f6e-8d19-38ab639a63a5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:12.791144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a8ffed09-9b29-4bb0-b781-0a470d878dd4 None None] MFA Rules not processed for user `c5e836103d274f9386f2dd933acd982d`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:12.793278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a8ffed09-9b29-4bb0-b781-0a470d878dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:12.841105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:12.842682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:12.842968 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:12.843394 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] PATH_INFO: `/v3/projects/5230f46303ec471bb9c78e0feaee0fc1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:12.845044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5230f46303ec471bb9c78e0feaee0fc1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:12.847106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:12.924701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5230f46303ec471bb9c78e0feaee0fc1', 'request_id': 'req-637cb213-33fd-4530-bb17-d40445cf63b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.930447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 857/1725] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:12.931794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffed09-9b29-4bb0-b781-0a470d878dd4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:12.937026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a8ffed09-9b29-4bb0-b781-0a470d878dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:12.944698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5230f46303ec471bb9c78e0feaee0fc1', 'request_id': 'req-637cb213-33fd-4530-bb17-d40445cf63b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:12.977655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5230f46303ec471bb9c78e0feaee0fc1', 'request_id': 'req-637cb213-33fd-4530-bb17-d40445cf63b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:13.000121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] The token cache is being invalidate because project 5230f46303ec471bb9c78e0feaee0fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:13.000603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:13.007194 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:13.013823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 869/1726] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:12 2022] DELETE /identity/v3/projects/5230f46303ec471bb9c78e0feaee0fc1 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:13.013823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:13.013823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:13.158146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cb213-33fd-4530-bb17-d40445cf63b0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:13.165332 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017f766-d5f0-4591-a134-e5c324a86ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:13.166704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017f766-d5f0-4591-a134-e5c324a86ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:13.166974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017f766-d5f0-4591-a134-e5c324a86ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:13.167253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017f766-d5f0-4591-a134-e5c324a86ba3 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:13.168800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017f766-d5f0-4591-a134-e5c324a86ba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:13.172662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017f766-d5f0-4591-a134-e5c324a86ba3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:13.187624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:13.216737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 870/1727] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:13 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:13.221538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017f766-d5f0-4591-a134-e5c324a86ba3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:13.222332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffed09-9b29-4bb0-b781-0a470d878dd4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:13.225182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5017f766-d5f0-4591-a134-e5c324a86ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:13.268046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017f766-d5f0-4591-a134-e5c324a86ba3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:13.283857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:13.284985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:13.285430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:13.289752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] PATH_INFO: `/v3/users/d2f3043aa48448d0b9cb355b46796f69` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:13.293759 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f3043aa48448d0b9cb355b46796f69)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:13.299259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:13.413333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f3043aa48448d0b9cb355b46796f69'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:13.420570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852d1a5-a2ed-4225-93e9-79c29954f241 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:13.421900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852d1a5-a2ed-4225-93e9-79c29954f241 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:13.422253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852d1a5-a2ed-4225-93e9-79c29954f241 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:13.422602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852d1a5-a2ed-4225-93e9-79c29954f241 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:13.425327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f3043aa48448d0b9cb355b46796f69'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:13.426292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f3043aa48448d0b9cb355b46796f69'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:13.439140 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8852d1a5-a2ed-4225-93e9-79c29954f241 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:13.450363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f3043aa48448d0b9cb355b46796f69'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:13.470889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:13.476098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e078547b-33f7-4256-8f81-fccf069e19b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f3043aa48448d0b9cb355b46796f69'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:13.497811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 871/1728] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:13 2022] DELETE /identity/v3/users/d2f3043aa48448d0b9cb355b46796f69 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:13.497811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a61a2-e08c-4fe3-9da3-88e1e2bee90d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:13.497811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a61a2-e08c-4fe3-9da3-88e1e2bee90d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:13.497811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a61a2-e08c-4fe3-9da3-88e1e2bee90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:13.566023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-894a61a2-e08c-4fe3-9da3-88e1e2bee90d None None] MFA Rules not processed for user `c5e836103d274f9386f2dd933acd982d`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:13.567437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-894a61a2-e08c-4fe3-9da3-88e1e2bee90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:13.642476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852d1a5-a2ed-4225-93e9-79c29954f241 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:13.649203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852d1a5-a2ed-4225-93e9-79c29954f241 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:13.672330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 858/1729] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:13.673374 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852d1a5-a2ed-4225-93e9-79c29954f241 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:13.680849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8852d1a5-a2ed-4225-93e9-79c29954f241 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:13.704825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852d1a5-a2ed-4225-93e9-79c29954f241 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:13.708228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 872/1730] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:13.709405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a61a2-e08c-4fe3-9da3-88e1e2bee90d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:13.714851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-894a61a2-e08c-4fe3-9da3-88e1e2bee90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:13.869209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:13.871270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:13.872465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:13.873037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] PATH_INFO: `/v3/users/760312586f42485ebe51b0c7130b4904` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:13.875128 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] RBAC: Authorizing `identity:delete_user(user_id=760312586f42485ebe51b0c7130b4904)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:13.877565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:13.888492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a61a2-e08c-4fe3-9da3-88e1e2bee90d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:13.897162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70df5d-7095-451f-ae90-5df1dd83e505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:13.898306 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70df5d-7095-451f-ae90-5df1dd83e505 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:13.898651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70df5d-7095-451f-ae90-5df1dd83e505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:13.899081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70df5d-7095-451f-ae90-5df1dd83e505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:13.900082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70df5d-7095-451f-ae90-5df1dd83e505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:13.903095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70df5d-7095-451f-ae90-5df1dd83e505 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:13.926155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 873/1731] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:13.927281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70df5d-7095-451f-ae90-5df1dd83e505 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:13.929458 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8d70df5d-7095-451f-ae90-5df1dd83e505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:13.988825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70df5d-7095-451f-ae90-5df1dd83e505 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:14.000658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:14.002185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:14.002475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:14.002755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] PATH_INFO: `/v3/projects/972624793e4b49d8a6c04f1d019649d7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:14.004478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=972624793e4b49d8a6c04f1d019649d7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:14.010875 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:14.013710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:14.065910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760312586f42485ebe51b0c7130b4904'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:14.067990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760312586f42485ebe51b0c7130b4904'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:14.069504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760312586f42485ebe51b0c7130b4904'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:14.090651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760312586f42485ebe51b0c7130b4904'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:14.129592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760312586f42485ebe51b0c7130b4904'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:14.161258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 859/1732] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:13 2022] DELETE /identity/v3/users/760312586f42485ebe51b0c7130b4904 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:14.162892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '972624793e4b49d8a6c04f1d019649d7', 'request_id': 'req-de35cc7b-23d0-4f21-996f-3963fe16b4ed'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:14.163559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:14.168370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:14.178708 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '972624793e4b49d8a6c04f1d019649d7', 'request_id': 'req-de35cc7b-23d0-4f21-996f-3963fe16b4ed'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:14.197578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '972624793e4b49d8a6c04f1d019649d7', 'request_id': 'req-de35cc7b-23d0-4f21-996f-3963fe16b4ed'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:14.225644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] The token cache is being invalidate because project 972624793e4b49d8a6c04f1d019649d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:14.225958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:14.227800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:14.241963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 874/1733] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:13 2022] DELETE /identity/v3/projects/972624793e4b49d8a6c04f1d019649d7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:14.242950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:14.244836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:14.278437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:14.377882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc9d1d-83a8-4740-b9d1-9344395693db admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:14.390454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a9860-6f05-4170-a2a6-7eccb130c995 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:14.393038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a9860-6f05-4170-a2a6-7eccb130c995 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:14.393331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a9860-6f05-4170-a2a6-7eccb130c995 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:14.393616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a9860-6f05-4170-a2a6-7eccb130c995 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:14.398009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35cc7b-23d0-4f21-996f-3963fe16b4ed admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:14.399984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c68a9860-6f05-4170-a2a6-7eccb130c995 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:14.535549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a9860-6f05-4170-a2a6-7eccb130c995 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:14.539642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a9860-6f05-4170-a2a6-7eccb130c995 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:14.568646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f16df7-2724-4835-9099-edf75d849075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:14.570323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f16df7-2724-4835-9099-edf75d849075 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:14.570499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f16df7-2724-4835-9099-edf75d849075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:14.570782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f16df7-2724-4835-9099-edf75d849075 admin admin] PATH_INFO: `/v3/projects/6c39ae1e47174687bfc8e8a1000cfe51/users/d847298e58234177bd921c7c78715ae7/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:14.576553 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 860/1734] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:14.609856 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e413c-535c-4e63-8dc7-5b379580e82b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:14.610165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e413c-535c-4e63-8dc7-5b379580e82b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:14.610446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e413c-535c-4e63-8dc7-5b379580e82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:14.666143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f16df7-2724-4835-9099-edf75d849075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c39ae1e47174687bfc8e8a1000cfe51, user_id=d847298e58234177bd921c7c78715ae7, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:14.677883 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-244e413c-535c-4e63-8dc7-5b379580e82b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:14.678339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f16df7-2724-4835-9099-edf75d849075 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:14.679138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-244e413c-535c-4e63-8dc7-5b379580e82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:14.695247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 861/1735] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:14.705025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8d027-7661-41c4-a622-64fd4acf17a2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:14.705317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8d027-7661-41c4-a622-64fd4acf17a2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:14.705598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8d027-7661-41c4-a622-64fd4acf17a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:14.749232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 875/1736] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:14 2022] PUT /identity/v3/projects/6c39ae1e47174687bfc8e8a1000cfe51/users/d847298e58234177bd921c7c78715ae7/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:14.767476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990aaef-a594-4002-aa13-9bc43ad394d1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:14.767476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990aaef-a594-4002-aa13-9bc43ad394d1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:14.767476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990aaef-a594-4002-aa13-9bc43ad394d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:14.786061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-95d8d027-7661-41c4-a622-64fd4acf17a2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:14.794112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-95d8d027-7661-41c4-a622-64fd4acf17a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:14.909524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c990aaef-a594-4002-aa13-9bc43ad394d1 None None] MFA Rules not processed for user `1bf6e7a4afaf405caae7d750973d9956`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:14.910823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c990aaef-a594-4002-aa13-9bc43ad394d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:14.935992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:14.978989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 862/1737] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:14.980766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8d027-7661-41c4-a622-64fd4acf17a2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:15.007133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8d027-7661-41c4-a622-64fd4acf17a2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:15.015949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc134676-cc11-409a-8d4a-e675c05c25e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:15.016475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc134676-cc11-409a-8d4a-e675c05c25e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.017208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc134676-cc11-409a-8d4a-e675c05c25e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.020630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc134676-cc11-409a-8d4a-e675c05c25e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:15.021496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc134676-cc11-409a-8d4a-e675c05c25e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:15.023317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc134676-cc11-409a-8d4a-e675c05c25e1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:15.059323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 863/1738] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:15.060961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a257d68-d9f8-47d7-8e8a-538734daecdc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.061255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a257d68-d9f8-47d7-8e8a-538734daecdc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.061538 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a257d68-d9f8-47d7-8e8a-538734daecdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:15.062818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 876/1739] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:15.081912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990aaef-a594-4002-aa13-9bc43ad394d1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:15.085704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c990aaef-a594-4002-aa13-9bc43ad394d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:15.111059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990aaef-a594-4002-aa13-9bc43ad394d1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:15.125178 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:15.126379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.126763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.127139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] PATH_INFO: `/v3/projects/f8520fd77c304eab9377377f7bfdf266` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:15.130571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8a257d68-d9f8-47d7-8e8a-538734daecdc None None] MFA Rules not processed for user `fa82bddf9aa0455797d42993fe850ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:15.131931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8520fd77c304eab9377377f7bfdf266)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:15.132056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8a257d68-d9f8-47d7-8e8a-538734daecdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:15.135483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:15.241486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8520fd77c304eab9377377f7bfdf266', 'request_id': 'req-a145c39c-0844-43b9-8b5e-ddc197e037b3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:15.242689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:15.255585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8520fd77c304eab9377377f7bfdf266', 'request_id': 'req-a145c39c-0844-43b9-8b5e-ddc197e037b3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:15.263322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 864/1740] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:15.263322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a257d68-d9f8-47d7-8e8a-538734daecdc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:15.280270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8520fd77c304eab9377377f7bfdf266', 'request_id': 'req-a145c39c-0844-43b9-8b5e-ddc197e037b3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:15.296699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] The token cache is being invalidate because project f8520fd77c304eab9377377f7bfdf266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:15.297571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:15.299571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:15.302938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 877/1741] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:15 2022] DELETE /identity/v3/projects/f8520fd77c304eab9377377f7bfdf266 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:15.305582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:15.310195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:15.314029 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a257d68-d9f8-47d7-8e8a-538734daecdc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:15.447432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145c39c-0844-43b9-8b5e-ddc197e037b3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:15.461000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d204d-6f5f-4493-9a34-e62e905cec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:15.461936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d204d-6f5f-4493-9a34-e62e905cec78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.462206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d204d-6f5f-4493-9a34-e62e905cec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.462636 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d204d-6f5f-4493-9a34-e62e905cec78 admin admin] PATH_INFO: `/v3/projects/6c39ae1e47174687bfc8e8a1000cfe51/users/d847298e58234177bd921c7c78715ae7/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:15.468757 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d204d-6f5f-4493-9a34-e62e905cec78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c39ae1e47174687bfc8e8a1000cfe51, user_id=d847298e58234177bd921c7c78715ae7, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:15.471580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d204d-6f5f-4493-9a34-e62e905cec78 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:15.486113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cb948-0212-4826-8164-a4fe1cf50d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:15.487848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cb948-0212-4826-8164-a4fe1cf50d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.488139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cb948-0212-4826-8164-a4fe1cf50d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.488468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cb948-0212-4826-8164-a4fe1cf50d5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:15.489225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cb948-0212-4826-8164-a4fe1cf50d5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:15.489544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cb948-0212-4826-8164-a4fe1cf50d5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:15.501141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cb948-0212-4826-8164-a4fe1cf50d5e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:15.501483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 878/1742] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:15 2022] PUT /identity/v3/projects/6c39ae1e47174687bfc8e8a1000cfe51/users/d847298e58234177bd921c7c78715ae7/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:15.502830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2bf45-4f43-41b4-bccc-85aac1807e30 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.505567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2bf45-4f43-41b4-bccc-85aac1807e30 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.505928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2bf45-4f43-41b4-bccc-85aac1807e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:15.550582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 865/1743] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:15.552055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5613d-80ee-424f-843d-d5e9ab0d3c20 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.552366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5613d-80ee-424f-843d-d5e9ab0d3c20 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.552654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5613d-80ee-424f-843d-d5e9ab0d3c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:15.560594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-48a2bf45-4f43-41b4-bccc-85aac1807e30 None None] MFA Rules not processed for user `1bf6e7a4afaf405caae7d750973d9956`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:15.561945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-48a2bf45-4f43-41b4-bccc-85aac1807e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:15.610162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a0d5613d-80ee-424f-843d-d5e9ab0d3c20 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:15.611556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a0d5613d-80ee-424f-843d-d5e9ab0d3c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:15.631317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:15.708273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 879/1744] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:15.711447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae7f0-ca86-4f92-9eca-af8aa6cce2ac None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.746408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae7f0-ca86-4f92-9eca-af8aa6cce2ac None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.748966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae7f0-ca86-4f92-9eca-af8aa6cce2ac None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:15.773291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 880/1745] 10.222.0.57 () {54 vars in 912 bytes} [Thu Jul 21 17:34:15 2022] GET /identity => generated 270 bytes in 64 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:34:15.784341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd30fd-c723-446a-9764-db922dd6eb7a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.784815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd30fd-c723-446a-9764-db922dd6eb7a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.786174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd30fd-c723-446a-9764-db922dd6eb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:15.791720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 866/1746] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:15.793680 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5613d-80ee-424f-843d-d5e9ab0d3c20 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:15.795184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a0d5613d-80ee-424f-843d-d5e9ab0d3c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:15.812796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5613d-80ee-424f-843d-d5e9ab0d3c20 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:15.828379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797ae21-b716-4b43-b0c7-1b0e6b39e5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:15.829335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797ae21-b716-4b43-b0c7-1b0e6b39e5a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.829335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797ae21-b716-4b43-b0c7-1b0e6b39e5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.829690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797ae21-b716-4b43-b0c7-1b0e6b39e5a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:15.830299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797ae21-b716-4b43-b0c7-1b0e6b39e5a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:15.832392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797ae21-b716-4b43-b0c7-1b0e6b39e5a3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:15.854805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e7dd30fd-c723-446a-9764-db922dd6eb7a None None] MFA Rules not processed for user `d847298e58234177bd921c7c78715ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:15.859099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e7dd30fd-c723-446a-9764-db922dd6eb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:15.905066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 867/1747] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:15 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:15.906783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9449b-fd24-4c0b-b3e5-9d997b0fd9e6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:15.907289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9449b-fd24-4c0b-b3e5-9d997b0fd9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:15.907942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9449b-fd24-4c0b-b3e5-9d997b0fd9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.016129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-46e9449b-fd24-4c0b-b3e5-9d997b0fd9e6 None None] MFA Rules not processed for user `b0edda384caf46819169b69b858afc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:16.018422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-46e9449b-fd24-4c0b-b3e5-9d997b0fd9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.062862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 881/1748] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:16.069179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338cc6-d7f1-46cc-83ca-b1d4a7c35bce None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.071484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338cc6-d7f1-46cc-83ca-b1d4a7c35bce None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.071814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338cc6-d7f1-46cc-83ca-b1d4a7c35bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.106232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-13338cc6-d7f1-46cc-83ca-b1d4a7c35bce None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:16.108006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-13338cc6-d7f1-46cc-83ca-b1d4a7c35bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.125572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 868/1749] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 17:34:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:16.126459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9449b-fd24-4c0b-b3e5-9d997b0fd9e6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:16.131272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-46e9449b-fd24-4c0b-b3e5-9d997b0fd9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.144247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:16.208633 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 882/1750] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:16.210101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13338cc6-d7f1-46cc-83ca-b1d4a7c35bce None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:16.225974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13338cc6-d7f1-46cc-83ca-b1d4a7c35bce None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:16.237267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc6515-4a36-4550-9c51-d0239b8d7087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:16.238324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc6515-4a36-4550-9c51-d0239b8d7087 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.240131 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc6515-4a36-4550-9c51-d0239b8d7087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.240657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc6515-4a36-4550-9c51-d0239b8d7087 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.243896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc6515-4a36-4550-9c51-d0239b8d7087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:16.258718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc6515-4a36-4550-9c51-d0239b8d7087 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:16.273658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9449b-fd24-4c0b-b3e5-9d997b0fd9e6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:16.288720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:16.289697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.289986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.294018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 883/1751] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:16 2022] POST /identity/v3/users => generated 290 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:16.294181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] PATH_INFO: `/v3/projects/f62992d517c74e68a8528e32345eab77` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.294259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae74972-b3bd-4da0-9ab9-5bcfedaabc7c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.294340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae74972-b3bd-4da0-9ab9-5bcfedaabc7c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.294609 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae74972-b3bd-4da0-9ab9-5bcfedaabc7c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.295879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f62992d517c74e68a8528e32345eab77)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:16.298464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:16.300936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 884/1752] 10.222.0.57 () {54 vars in 912 bytes} [Thu Jul 21 17:34:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:34:16.301564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae74972-b3bd-4da0-9ab9-5bcfedaabc7c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:16.303424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4ae74972-b3bd-4da0-9ab9-5bcfedaabc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.381324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f62992d517c74e68a8528e32345eab77', 'request_id': 'req-45cef368-cc43-4156-aeaa-6d29c61b8883'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:16.396267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f62992d517c74e68a8528e32345eab77', 'request_id': 'req-45cef368-cc43-4156-aeaa-6d29c61b8883'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:16.410761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:16.412992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f62992d517c74e68a8528e32345eab77', 'request_id': 'req-45cef368-cc43-4156-aeaa-6d29c61b8883'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:16.441729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae74972-b3bd-4da0-9ab9-5bcfedaabc7c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:16.442839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] The token cache is being invalidate because project f62992d517c74e68a8528e32345eab77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:16.443215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:16.444054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:16.455389 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 869/1753] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:16 2022] DELETE /identity/v3/projects/f62992d517c74e68a8528e32345eab77 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:16.460284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:16.462952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.567981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07a250-77eb-4f61-8034-3f24007b5b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:16.568687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07a250-77eb-4f61-8034-3f24007b5b81 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.568980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07a250-77eb-4f61-8034-3f24007b5b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.569263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07a250-77eb-4f61-8034-3f24007b5b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.574902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ab07a250-77eb-4f61-8034-3f24007b5b81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.604062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cef368-cc43-4156-aeaa-6d29c61b8883 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:16.615570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60124b6-b59f-4242-980b-5e6b4aad932e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:16.616253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60124b6-b59f-4242-980b-5e6b4aad932e service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.616822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60124b6-b59f-4242-980b-5e6b4aad932e service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.617108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60124b6-b59f-4242-980b-5e6b4aad932e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.622509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a60124b6-b59f-4242-980b-5e6b4aad932e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.648980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60124b6-b59f-4242-980b-5e6b4aad932e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:16.652151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60124b6-b59f-4242-980b-5e6b4aad932e service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:16.679680 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 870/1754] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:16.681013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60124b6-b59f-4242-980b-5e6b4aad932e service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:16.684421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a60124b6-b59f-4242-980b-5e6b4aad932e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.752381 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:16.771814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07a250-77eb-4f61-8034-3f24007b5b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:16.778096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07a250-77eb-4f61-8034-3f24007b5b81 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:16.792707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60124b6-b59f-4242-980b-5e6b4aad932e service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:16.800054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862b3f8-64a6-4ce2-90f3-edbc7b8ebef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:16.801634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862b3f8-64a6-4ce2-90f3-edbc7b8ebef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.802099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862b3f8-64a6-4ce2-90f3-edbc7b8ebef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.802426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862b3f8-64a6-4ce2-90f3-edbc7b8ebef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.804324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862b3f8-64a6-4ce2-90f3-edbc7b8ebef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:16.804556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862b3f8-64a6-4ce2-90f3-edbc7b8ebef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:16.805090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 885/1755] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:16.805769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07a250-77eb-4f61-8034-3f24007b5b81 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:16.806180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862b3f8-64a6-4ce2-90f3-edbc7b8ebef8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:16.814071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ab07a250-77eb-4f61-8034-3f24007b5b81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.825278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 871/1756] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:16.827204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70a592-652b-4e91-8754-b690ef197eea None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.827204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70a592-652b-4e91-8754-b690ef197eea None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.827485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70a592-652b-4e91-8754-b690ef197eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.841139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07a250-77eb-4f61-8034-3f24007b5b81 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:16.852196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5c70a592-652b-4e91-8754-b690ef197eea None None] MFA Rules not processed for user `b0edda384caf46819169b69b858afc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:16.853489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5c70a592-652b-4e91-8754-b690ef197eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.857938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9c72d-32b1-432a-9237-39cf727ab76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:16.858628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9c72d-32b1-432a-9237-39cf727ab76e admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.858913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9c72d-32b1-432a-9237-39cf727ab76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.859285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9c72d-32b1-432a-9237-39cf727ab76e admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.860224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9c72d-32b1-432a-9237-39cf727ab76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:16.862101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9c72d-32b1-432a-9237-39cf727ab76e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:16.863198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 872/1757] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 17:34:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:16.865303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f54fe-ee3c-40b5-963f-d53224c20f85 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.865597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f54fe-ee3c-40b5-963f-d53224c20f85 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.865884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f54fe-ee3c-40b5-963f-d53224c20f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.889904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fe1f54fe-ee3c-40b5-963f-d53224c20f85 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:16.891145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fe1f54fe-ee3c-40b5-963f-d53224c20f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:16.897002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 886/1758] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:16.898165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9c72d-32b1-432a-9237-39cf727ab76e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:16.916468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 873/1759] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:16.918005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f54fe-ee3c-40b5-963f-d53224c20f85 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:16.937007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f54fe-ee3c-40b5-963f-d53224c20f85 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:16.941813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9c72d-32b1-432a-9237-39cf727ab76e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:16.943246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddd90a-458d-4a47-9281-e10a5bfe6c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:16.943925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddd90a-458d-4a47-9281-e10a5bfe6c18 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.945102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddd90a-458d-4a47-9281-e10a5bfe6c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.945508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddd90a-458d-4a47-9281-e10a5bfe6c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.954493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080eacf-eca2-4dda-9dce-87ad0e944ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:16.955203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080eacf-eca2-4dda-9dce-87ad0e944ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:16.955875 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080eacf-eca2-4dda-9dce-87ad0e944ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:16.956657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080eacf-eca2-4dda-9dce-87ad0e944ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:16.959208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddd90a-458d-4a47-9281-e10a5bfe6c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:16.960058 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080eacf-eca2-4dda-9dce-87ad0e944ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:16.961678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddd90a-458d-4a47-9281-e10a5bfe6c18 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:16.964054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080eacf-eca2-4dda-9dce-87ad0e944ae9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:16.973635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 874/1760] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:16.974568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddd90a-458d-4a47-9281-e10a5bfe6c18 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:16.994357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddd90a-458d-4a47-9281-e10a5bfe6c18 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:17.000486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9a272-6432-4507-9d18-6ded22f399d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:17.001235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9a272-6432-4507-9d18-6ded22f399d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:17.001524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9a272-6432-4507-9d18-6ded22f399d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:17.001807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9a272-6432-4507-9d18-6ded22f399d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:17.003178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9a272-6432-4507-9d18-6ded22f399d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:17.004898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9a272-6432-4507-9d18-6ded22f399d1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:17.022170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 875/1761] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:17.023687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ceca3-3a31-44ec-b361-c207845b7366 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:17.024042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ceca3-3a31-44ec-b361-c207845b7366 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:17.024369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ceca3-3a31-44ec-b361-c207845b7366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:17.051915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 887/1762] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:16 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:17.051915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080eacf-eca2-4dda-9dce-87ad0e944ae9 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:17.054337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4080eacf-eca2-4dda-9dce-87ad0e944ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:17.061831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c77ceca3-3a31-44ec-b361-c207845b7366 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:17.069122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c77ceca3-3a31-44ec-b361-c207845b7366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:17.216160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 876/1763] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:17.216160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ceca3-3a31-44ec-b361-c207845b7366 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:17.246073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ceca3-3a31-44ec-b361-c207845b7366 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:17.253233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51390f-5089-4d30-a35d-899125c6fb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:17.254002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51390f-5089-4d30-a35d-899125c6fb28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:17.254282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51390f-5089-4d30-a35d-899125c6fb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:17.254560 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51390f-5089-4d30-a35d-899125c6fb28 admin admin] PATH_INFO: `/v3/projects/f5edf868e94648bb91a11134ddd09b52/users/9d8ea15803c647dca00295f3979ecb53/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:17.285544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51390f-5089-4d30-a35d-899125c6fb28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5edf868e94648bb91a11134ddd09b52, user_id=9d8ea15803c647dca00295f3979ecb53, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:17.288948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51390f-5089-4d30-a35d-899125c6fb28 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:17.338112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 877/1764] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:17 2022] PUT /identity/v3/projects/f5edf868e94648bb91a11134ddd09b52/users/9d8ea15803c647dca00295f3979ecb53/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:17.338112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51390f-5089-4d30-a35d-899125c6fb28 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:17.340099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080eacf-eca2-4dda-9dce-87ad0e944ae9 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:17.415415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:17.488907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541f884-c62b-4cbf-bf3b-faea8e8ce767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:17.488907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541f884-c62b-4cbf-bf3b-faea8e8ce767 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:17.488907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541f884-c62b-4cbf-bf3b-faea8e8ce767 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:17.489265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541f884-c62b-4cbf-bf3b-faea8e8ce767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:17.494442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4541f884-c62b-4cbf-bf3b-faea8e8ce767 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:17.505974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51390f-5089-4d30-a35d-899125c6fb28 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:17.518418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec68d5d-d3e7-46b8-b979-da72a6758fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:17.520083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec68d5d-d3e7-46b8-b979-da72a6758fff admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:17.520877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec68d5d-d3e7-46b8-b979-da72a6758fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:17.521356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec68d5d-d3e7-46b8-b979-da72a6758fff admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:17.522252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec68d5d-d3e7-46b8-b979-da72a6758fff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:17.527497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec68d5d-d3e7-46b8-b979-da72a6758fff admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:17.606419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:17.628709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 878/1765] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:17 2022] POST /identity/v3/users => generated 283 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:17.630063 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec68d5d-d3e7-46b8-b979-da72a6758fff admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:17.661820 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec68d5d-d3e7-46b8-b979-da72a6758fff admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:17.677866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253d94a-9305-4127-a6c9-010e7f1cd66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:17.681461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253d94a-9305-4127-a6c9-010e7f1cd66e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:17.683268 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541f884-c62b-4cbf-bf3b-faea8e8ce767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:17.683626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253d94a-9305-4127-a6c9-010e7f1cd66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:17.684080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253d94a-9305-4127-a6c9-010e7f1cd66e admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:17.685071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253d94a-9305-4127-a6c9-010e7f1cd66e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:17.687447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253d94a-9305-4127-a6c9-010e7f1cd66e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:17.687986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541f884-c62b-4cbf-bf3b-faea8e8ce767 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:17.692484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253d94a-9305-4127-a6c9-010e7f1cd66e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:17.727685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 879/1766] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:17.729232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227694ef-92a7-4288-92a0-2f032cd9316e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:17.729524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227694ef-92a7-4288-92a0-2f032cd9316e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:17.730687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 888/1767] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:17 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:17.731622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541f884-c62b-4cbf-bf3b-faea8e8ce767 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:17.732398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227694ef-92a7-4288-92a0-2f032cd9316e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:17.763874 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541f884-c62b-4cbf-bf3b-faea8e8ce767 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:17.785114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb881930-09fa-4fc8-8cfd-cbdc0f18315a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:17.785781 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb881930-09fa-4fc8-8cfd-cbdc0f18315a admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:17.786164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb881930-09fa-4fc8-8cfd-cbdc0f18315a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:17.786274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb881930-09fa-4fc8-8cfd-cbdc0f18315a admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:17.787213 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb881930-09fa-4fc8-8cfd-cbdc0f18315a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:17.788674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb881930-09fa-4fc8-8cfd-cbdc0f18315a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:17.811588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-227694ef-92a7-4288-92a0-2f032cd9316e None None] MFA Rules not processed for user `9d8ea15803c647dca00295f3979ecb53`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:17.811588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-227694ef-92a7-4288-92a0-2f032cd9316e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:17.838492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 889/1768] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:17.838492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb881930-09fa-4fc8-8cfd-cbdc0f18315a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:17.886122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb881930-09fa-4fc8-8cfd-cbdc0f18315a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:17.903100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fa76b-5677-48db-a788-20f7bc9a2d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:17.905480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fa76b-5677-48db-a788-20f7bc9a2d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:17.906913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fa76b-5677-48db-a788-20f7bc9a2d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:17.907212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fa76b-5677-48db-a788-20f7bc9a2d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:17.916536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fa76b-5677-48db-a788-20f7bc9a2d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:17.922784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fa76b-5677-48db-a788-20f7bc9a2d2a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:18.025292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 890/1769] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:17 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:18.026531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fa76b-5677-48db-a788-20f7bc9a2d2a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:18.102231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 880/1770] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:18.102859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227694ef-92a7-4288-92a0-2f032cd9316e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:18.112767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-227694ef-92a7-4288-92a0-2f032cd9316e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:18.268084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fa76b-5677-48db-a788-20f7bc9a2d2a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:18.286541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23418e70-6368-48fb-a673-219b050e8833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:18.287288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23418e70-6368-48fb-a673-219b050e8833 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:18.288407 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23418e70-6368-48fb-a673-219b050e8833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:18.288690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23418e70-6368-48fb-a673-219b050e8833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:18.289737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23418e70-6368-48fb-a673-219b050e8833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:18.293198 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23418e70-6368-48fb-a673-219b050e8833 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:18.333354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 891/1771] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:18.333826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23418e70-6368-48fb-a673-219b050e8833 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:18.368496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23418e70-6368-48fb-a673-219b050e8833 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:18.387367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b22ec0-5395-4e77-a3a1-34c5ef1321d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:18.388064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b22ec0-5395-4e77-a3a1-34c5ef1321d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:18.390583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b22ec0-5395-4e77-a3a1-34c5ef1321d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:18.390873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b22ec0-5395-4e77-a3a1-34c5ef1321d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:18.391597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b22ec0-5395-4e77-a3a1-34c5ef1321d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:18.397809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b22ec0-5395-4e77-a3a1-34c5ef1321d6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:18.399219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227694ef-92a7-4288-92a0-2f032cd9316e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:18.456599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 892/1772] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:18 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:18.456599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b22ec0-5395-4e77-a3a1-34c5ef1321d6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:18.488171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b22ec0-5395-4e77-a3a1-34c5ef1321d6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:18.503408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e790e-4564-433f-8755-67e58d493770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:18.504094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e790e-4564-433f-8755-67e58d493770 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:18.504409 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e790e-4564-433f-8755-67e58d493770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:18.504684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e790e-4564-433f-8755-67e58d493770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:18.509092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e790e-4564-433f-8755-67e58d493770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:18.513070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e790e-4564-433f-8755-67e58d493770 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:18.552255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668ea13-be64-4e93-a35c-de3e64f29e7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:18.553207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668ea13-be64-4e93-a35c-de3e64f29e7f service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:18.553498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668ea13-be64-4e93-a35c-de3e64f29e7f service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:18.553818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668ea13-be64-4e93-a35c-de3e64f29e7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:18.556014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4668ea13-be64-4e93-a35c-de3e64f29e7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:18.581293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 893/1773] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:18 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:18.582271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e790e-4564-433f-8755-67e58d493770 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:18.584128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-be4e790e-4564-433f-8755-67e58d493770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:18.584620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:18.599089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:18.720484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668ea13-be64-4e93-a35c-de3e64f29e7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:18.722438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668ea13-be64-4e93-a35c-de3e64f29e7f service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:18.728461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e790e-4564-433f-8755-67e58d493770 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:18.734491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 881/1774] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:18.735718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668ea13-be64-4e93-a35c-de3e64f29e7f service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:18.735871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e93245-9714-43c0-b739-a286ead0bf7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:18.736674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e93245-9714-43c0-b739-a286ead0bf7a service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:18.737056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e93245-9714-43c0-b739-a286ead0bf7a service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:18.737663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4668ea13-be64-4e93-a35c-de3e64f29e7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:18.738023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e93245-9714-43c0-b739-a286ead0bf7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:18.740126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-08e93245-9714-43c0-b739-a286ead0bf7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:18.755748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e93245-9714-43c0-b739-a286ead0bf7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:18.758766 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e93245-9714-43c0-b739-a286ead0bf7a service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:18.776101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 894/1775] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:18.778731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e93245-9714-43c0-b739-a286ead0bf7a service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:18.859332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668ea13-be64-4e93-a35c-de3e64f29e7f service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:18.866122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced65ef3-68b5-4379-b715-b5e309d71c76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:18.866941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced65ef3-68b5-4379-b715-b5e309d71c76 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:18.867324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced65ef3-68b5-4379-b715-b5e309d71c76 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:18.867730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced65ef3-68b5-4379-b715-b5e309d71c76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:18.880074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced65ef3-68b5-4379-b715-b5e309d71c76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:18.882132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced65ef3-68b5-4379-b715-b5e309d71c76 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:18.896422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 882/1776] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:18.897572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced65ef3-68b5-4379-b715-b5e309d71c76 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:18.970699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e93245-9714-43c0-b739-a286ead0bf7a service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:18.979644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888e52b-5173-40ca-878d-14be098303f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:18.986993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888e52b-5173-40ca-878d-14be098303f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:18.987286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888e52b-5173-40ca-878d-14be098303f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:18.987703 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888e52b-5173-40ca-878d-14be098303f2 admin admin] PATH_INFO: `/v3/projects/cf2f3b226528454bba5168b0a972f4d0/users/5ac91a12c8fd427092042eced55fc61f/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.012874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced65ef3-68b5-4379-b715-b5e309d71c76 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:19.019972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c888e52b-5173-40ca-878d-14be098303f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf2f3b226528454bba5168b0a972f4d0, user_id=5ac91a12c8fd427092042eced55fc61f, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:19.023788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c888e52b-5173-40ca-878d-14be098303f2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:19.026297 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5e40-28c9-47b6-a12e-95f24708f3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:19.028617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5e40-28c9-47b6-a12e-95f24708f3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:19.028997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5e40-28c9-47b6-a12e-95f24708f3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:19.029363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5e40-28c9-47b6-a12e-95f24708f3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.030226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b5e40-28c9-47b6-a12e-95f24708f3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:19.032811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b5e40-28c9-47b6-a12e-95f24708f3c2 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:19.058566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 895/1777] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:18 2022] PUT /identity/v3/projects/cf2f3b226528454bba5168b0a972f4d0/users/5ac91a12c8fd427092042eced55fc61f/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:19.069487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888e52b-5173-40ca-878d-14be098303f2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:19.072023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 883/1778] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:19.081928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6437a15-8ff4-4e1b-956d-b3dafb42621c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:19.083572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6437a15-8ff4-4e1b-956d-b3dafb42621c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:19.085383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6437a15-8ff4-4e1b-956d-b3dafb42621c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.127625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c6437a15-8ff4-4e1b-956d-b3dafb42621c None None] MFA Rules not processed for user `2ea3ea36439143b8a55f181170efc14a`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:19.128889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c6437a15-8ff4-4e1b-956d-b3dafb42621c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:19.193072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:19.236098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888e52b-5173-40ca-878d-14be098303f2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:19.247391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e405c02-835b-44ed-be29-fbf46a2d915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:19.247995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e405c02-835b-44ed-be29-fbf46a2d915c admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:19.248357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e405c02-835b-44ed-be29-fbf46a2d915c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:19.249243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e405c02-835b-44ed-be29-fbf46a2d915c admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.250546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e405c02-835b-44ed-be29-fbf46a2d915c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:19.254065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e405c02-835b-44ed-be29-fbf46a2d915c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:19.297776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 896/1779] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:19 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:19.303254 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4876dc-c606-4f65-a7f6-5b5c703a5202 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:19.303742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4876dc-c606-4f65-a7f6-5b5c703a5202 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:19.304033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4876dc-c606-4f65-a7f6-5b5c703a5202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.306683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 884/1780] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:19.313412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6437a15-8ff4-4e1b-956d-b3dafb42621c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:19.349789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6437a15-8ff4-4e1b-956d-b3dafb42621c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:19.362386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:19.367235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4ead3-740b-4e11-a481-d4acbbb06dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:19.368146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4ead3-740b-4e11-a481-d4acbbb06dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:19.368465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4ead3-740b-4e11-a481-d4acbbb06dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:19.368800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4ead3-740b-4e11-a481-d4acbbb06dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.369651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4ead3-740b-4e11-a481-d4acbbb06dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:19.375264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4ead3-740b-4e11-a481-d4acbbb06dd1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:19.424886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 885/1781] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:19.426054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4ead3-740b-4e11-a481-d4acbbb06dd1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:19.435143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2a4876dc-c606-4f65-a7f6-5b5c703a5202 None None] MFA Rules not processed for user `5ac91a12c8fd427092042eced55fc61f`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:19.437592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2a4876dc-c606-4f65-a7f6-5b5c703a5202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:19.487134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4ead3-740b-4e11-a481-d4acbbb06dd1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:19.510472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fdbc6-886a-4e99-b4af-09e3ebf744df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:19.512198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fdbc6-886a-4e99-b4af-09e3ebf744df admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:19.512519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fdbc6-886a-4e99-b4af-09e3ebf744df admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:19.512810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fdbc6-886a-4e99-b4af-09e3ebf744df admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.513554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fdbc6-886a-4e99-b4af-09e3ebf744df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:19.533810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fdbc6-886a-4e99-b4af-09e3ebf744df admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:19.566148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 886/1782] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:19.568995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fdbc6-886a-4e99-b4af-09e3ebf744df admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:19.592637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 897/1783] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:19.593626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4876dc-c606-4f65-a7f6-5b5c703a5202 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:19.601396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fdbc6-886a-4e99-b4af-09e3ebf744df admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:19.610118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c433da-41b9-42fc-9592-1b972c308d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:19.611223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c433da-41b9-42fc-9592-1b972c308d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:19.611640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c433da-41b9-42fc-9592-1b972c308d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:19.612073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c433da-41b9-42fc-9592-1b972c308d6f admin admin] PATH_INFO: `/v3/projects/29cbddf77dd9434caf011384f338bfcf/users/99ae98d641654d5ca21f50c8540981cf/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.613431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4876dc-c606-4f65-a7f6-5b5c703a5202 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:19.625044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628ee9f-deac-4d9a-b040-5fee4bac2355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:19.625729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628ee9f-deac-4d9a-b040-5fee4bac2355 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:19.626010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628ee9f-deac-4d9a-b040-5fee4bac2355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:19.626284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628ee9f-deac-4d9a-b040-5fee4bac2355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.629861 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628ee9f-deac-4d9a-b040-5fee4bac2355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:19.637017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628ee9f-deac-4d9a-b040-5fee4bac2355 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:19.652474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c433da-41b9-42fc-9592-1b972c308d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=29cbddf77dd9434caf011384f338bfcf, user_id=99ae98d641654d5ca21f50c8540981cf, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:19.655922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c433da-41b9-42fc-9592-1b972c308d6f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:19.673927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 898/1784] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:19.674954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628ee9f-deac-4d9a-b040-5fee4bac2355 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:19.701923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 887/1785] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:19 2022] PUT /identity/v3/projects/29cbddf77dd9434caf011384f338bfcf/users/99ae98d641654d5ca21f50c8540981cf/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:19.702948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c433da-41b9-42fc-9592-1b972c308d6f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:19.704231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628ee9f-deac-4d9a-b040-5fee4bac2355 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:19.878705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c433da-41b9-42fc-9592-1b972c308d6f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:19.904066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f125a0-b5a9-463c-91c2-2f69f9da837a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:19.905317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f125a0-b5a9-463c-91c2-2f69f9da837a admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:19.905605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f125a0-b5a9-463c-91c2-2f69f9da837a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:19.905922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f125a0-b5a9-463c-91c2-2f69f9da837a admin admin] PATH_INFO: `/v3/projects/22100922edd44f999f642bded1664109/users/89e4c7034cfb46f0a8ba0fd4e6135e17/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.912766 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e85622-750e-4453-9fe8-1e6653fc0644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:19.913535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e85622-750e-4453-9fe8-1e6653fc0644 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:19.913818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e85622-750e-4453-9fe8-1e6653fc0644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:19.914094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e85622-750e-4453-9fe8-1e6653fc0644 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:19.915117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e85622-750e-4453-9fe8-1e6653fc0644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:19.923590 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e85622-750e-4453-9fe8-1e6653fc0644 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:19.976577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f125a0-b5a9-463c-91c2-2f69f9da837a admin admin] RBAC: Authorizing `identity:create_grant(project_id=22100922edd44f999f642bded1664109, user_id=89e4c7034cfb46f0a8ba0fd4e6135e17, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:19.989848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f125a0-b5a9-463c-91c2-2f69f9da837a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:20.009686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 899/1786] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:19 2022] POST /identity/v3/projects => generated 363 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:20.014210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7599ec-2f71-4444-b81a-9f1b56f306c9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:20.015601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7599ec-2f71-4444-b81a-9f1b56f306c9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:20.017441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7599ec-2f71-4444-b81a-9f1b56f306c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:20.044608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 888/1787] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:19 2022] PUT /identity/v3/projects/22100922edd44f999f642bded1664109/users/89e4c7034cfb46f0a8ba0fd4e6135e17/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:20.046100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f125a0-b5a9-463c-91c2-2f69f9da837a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:20.114586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4c7599ec-2f71-4444-b81a-9f1b56f306c9 None None] MFA Rules not processed for user `99ae98d641654d5ca21f50c8540981cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:20.122860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4c7599ec-2f71-4444-b81a-9f1b56f306c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:20.297599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f125a0-b5a9-463c-91c2-2f69f9da837a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:20.326686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 900/1788] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:20.326686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d80c6-3a4f-4bb8-ab50-ba93272ec5f9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:20.326686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d80c6-3a4f-4bb8-ab50-ba93272ec5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:20.326686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d80c6-3a4f-4bb8-ab50-ba93272ec5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:20.336424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e883a-fbd8-4ff6-b639-2745f7fc06f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:20.337190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e883a-fbd8-4ff6-b639-2745f7fc06f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:20.337477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e883a-fbd8-4ff6-b639-2745f7fc06f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:20.337804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e883a-fbd8-4ff6-b639-2745f7fc06f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:20.341989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e883a-fbd8-4ff6-b639-2745f7fc06f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:20.344341 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e883a-fbd8-4ff6-b639-2745f7fc06f6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:20.345011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:20.405124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 889/1789] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:20 2022] POST /identity/v3/users => generated 288 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:20.406043 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e883a-fbd8-4ff6-b639-2745f7fc06f6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:20.414657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-711d80c6-3a4f-4bb8-ab50-ba93272ec5f9 None None] MFA Rules not processed for user `89e4c7034cfb46f0a8ba0fd4e6135e17`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:20.418177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-711d80c6-3a4f-4bb8-ab50-ba93272ec5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:20.435506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e883a-fbd8-4ff6-b639-2745f7fc06f6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:20.462256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838afb5-4395-4af6-aef5-b9a4c7436e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:20.467968 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838afb5-4395-4af6-aef5-b9a4c7436e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:20.468469 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838afb5-4395-4af6-aef5-b9a4c7436e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:20.468757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838afb5-4395-4af6-aef5-b9a4c7436e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:20.469602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838afb5-4395-4af6-aef5-b9a4c7436e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:20.482166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838afb5-4395-4af6-aef5-b9a4c7436e9d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:20.490483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:20.578839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 890/1790] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:20 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:20.582705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838afb5-4395-4af6-aef5-b9a4c7436e9d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:20.583753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 901/1791] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:20.592858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d80c6-3a4f-4bb8-ab50-ba93272ec5f9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:20.626115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d80c6-3a4f-4bb8-ab50-ba93272ec5f9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:20.781959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dc2ad-3929-46e9-acf7-2de9116d41c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:20.783051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dc2ad-3929-46e9-acf7-2de9116d41c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:20.783537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dc2ad-3929-46e9-acf7-2de9116d41c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:20.784015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dc2ad-3929-46e9-acf7-2de9116d41c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:20.797418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838afb5-4395-4af6-aef5-b9a4c7436e9d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:20.799027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dc2ad-3929-46e9-acf7-2de9116d41c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:20.801946 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dc2ad-3929-46e9-acf7-2de9116d41c0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:20.810837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb83c8-9e56-4aee-a633-c41b20ce3076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:20.812404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb83c8-9e56-4aee-a633-c41b20ce3076 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:20.812518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb83c8-9e56-4aee-a633-c41b20ce3076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:20.813068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb83c8-9e56-4aee-a633-c41b20ce3076 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:20.813463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb83c8-9e56-4aee-a633-c41b20ce3076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:20.815089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb83c8-9e56-4aee-a633-c41b20ce3076 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:20.838810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 891/1792] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:20.844743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb83c8-9e56-4aee-a633-c41b20ce3076 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:20.849044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 902/1793] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:20 2022] POST /identity/v3/users => generated 299 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:20.868873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dc2ad-3929-46e9-acf7-2de9116d41c0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:20.879579 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb83c8-9e56-4aee-a633-c41b20ce3076 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:20.886957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914e415-8729-4373-a351-a9f667e6ac15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:20.888330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914e415-8729-4373-a351-a9f667e6ac15 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:20.888689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914e415-8729-4373-a351-a9f667e6ac15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:20.889037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914e415-8729-4373-a351-a9f667e6ac15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:20.890507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914e415-8729-4373-a351-a9f667e6ac15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:20.892589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914e415-8729-4373-a351-a9f667e6ac15 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:20.914688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dc2ad-3929-46e9-acf7-2de9116d41c0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:20.931163 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a81fa-f8dc-4138-875b-b1a18dc160f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:20.936964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a81fa-f8dc-4138-875b-b1a18dc160f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:20.937082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a81fa-f8dc-4138-875b-b1a18dc160f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:20.938400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a81fa-f8dc-4138-875b-b1a18dc160f6 admin admin] PATH_INFO: `/v3/projects/36ee5b84d1a04a39a7bcc218c327d13a/users/d10980e9d3b94047aa81a8646c91c336/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:20.983241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 892/1794] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:20 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:20.985669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914e415-8729-4373-a351-a9f667e6ac15 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:21.007785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a81fa-f8dc-4138-875b-b1a18dc160f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ee5b84d1a04a39a7bcc218c327d13a, user_id=d10980e9d3b94047aa81a8646c91c336, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:21.020042 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a81fa-f8dc-4138-875b-b1a18dc160f6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:21.063386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 903/1795] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:20 2022] PUT /identity/v3/projects/36ee5b84d1a04a39a7bcc218c327d13a/users/d10980e9d3b94047aa81a8646c91c336/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:21.071201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a81fa-f8dc-4138-875b-b1a18dc160f6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:21.172490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914e415-8729-4373-a351-a9f667e6ac15 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:21.241880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:21.251846 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a81fa-f8dc-4138-875b-b1a18dc160f6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:21.266303 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8da0f-c634-4b83-8d6a-9f1b675e71b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:21.267005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8da0f-c634-4b83-8d6a-9f1b675e71b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:21.267291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8da0f-c634-4b83-8d6a-9f1b675e71b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:21.267570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8da0f-c634-4b83-8d6a-9f1b675e71b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:21.268265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8da0f-c634-4b83-8d6a-9f1b675e71b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:21.282713 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8da0f-c634-4b83-8d6a-9f1b675e71b2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:21.331177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1429a-87fb-4b0c-8e7e-7172796e85ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:21.331973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1429a-87fb-4b0c-8e7e-7172796e85ff admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:21.333974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1429a-87fb-4b0c-8e7e-7172796e85ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:21.334194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1429a-87fb-4b0c-8e7e-7172796e85ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:21.336797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1429a-87fb-4b0c-8e7e-7172796e85ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:21.340587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1429a-87fb-4b0c-8e7e-7172796e85ff admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:21.351691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 904/1796] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:21 2022] POST /identity/v3/users => generated 295 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:21.358261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8da0f-c634-4b83-8d6a-9f1b675e71b2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:21.374308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 893/1797] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:21.374308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1429a-87fb-4b0c-8e7e-7172796e85ff admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:21.388424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:21.388954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8da0f-c634-4b83-8d6a-9f1b675e71b2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:21.397166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554a5a6-72a5-4581-81ab-69621b3a2616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:21.398486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554a5a6-72a5-4581-81ab-69621b3a2616 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:21.399135 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554a5a6-72a5-4581-81ab-69621b3a2616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:21.399622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554a5a6-72a5-4581-81ab-69621b3a2616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:21.400645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554a5a6-72a5-4581-81ab-69621b3a2616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:21.402601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554a5a6-72a5-4581-81ab-69621b3a2616 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:21.407159 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1429a-87fb-4b0c-8e7e-7172796e85ff admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:21.419752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 905/1798] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:21.421018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554a5a6-72a5-4581-81ab-69621b3a2616 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:21.426783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c96871-e403-43e1-af61-db46df79cd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:21.429800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c96871-e403-43e1-af61-db46df79cd40 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:21.430915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c96871-e403-43e1-af61-db46df79cd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:21.431222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c96871-e403-43e1-af61-db46df79cd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:21.435280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554a5a6-72a5-4581-81ab-69621b3a2616 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:21.438354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c96871-e403-43e1-af61-db46df79cd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:21.440974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d534d1-5db0-4115-ac1c-a35ad9ffae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:21.441118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c96871-e403-43e1-af61-db46df79cd40 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:21.443257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d534d1-5db0-4115-ac1c-a35ad9ffae14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:21.444464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d534d1-5db0-4115-ac1c-a35ad9ffae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:21.445018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d534d1-5db0-4115-ac1c-a35ad9ffae14 admin admin] PATH_INFO: `/v3/projects/e8ab6188ef1c4297b52720d0359b6ae8/users/49522fcc32564bdbbbefd20dc2b388a6/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:21.466968 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d534d1-5db0-4115-ac1c-a35ad9ffae14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8ab6188ef1c4297b52720d0359b6ae8, user_id=49522fcc32564bdbbbefd20dc2b388a6, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:21.470772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d534d1-5db0-4115-ac1c-a35ad9ffae14 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:21.501573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 894/1799] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:21.506079 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c96871-e403-43e1-af61-db46df79cd40 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:21.525321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 906/1800] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:21 2022] PUT /identity/v3/projects/e8ab6188ef1c4297b52720d0359b6ae8/users/49522fcc32564bdbbbefd20dc2b388a6/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:21.529055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d534d1-5db0-4115-ac1c-a35ad9ffae14 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:21.559525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c96871-e403-43e1-af61-db46df79cd40 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:21.729448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d534d1-5db0-4115-ac1c-a35ad9ffae14 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:21.748935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2c27b-f52f-4702-9c4e-be995dbaf1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:21.751094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2c27b-f52f-4702-9c4e-be995dbaf1c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:21.752006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2c27b-f52f-4702-9c4e-be995dbaf1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:21.753012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2c27b-f52f-4702-9c4e-be995dbaf1c5 admin admin] PATH_INFO: `/v3/projects/9fd6cd4d3ca7423ca5490797475264f4/users/021d7fdc72b849668ea3765c5767c109/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:21.825001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dab778-50b7-4aa9-bfff-05dabec31e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:21.826077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dab778-50b7-4aa9-bfff-05dabec31e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:21.830906 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dab778-50b7-4aa9-bfff-05dabec31e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:21.831317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dab778-50b7-4aa9-bfff-05dabec31e74 admin admin] PATH_INFO: `/v3/projects/36ee5b84d1a04a39a7bcc218c327d13a/users/d10980e9d3b94047aa81a8646c91c336/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:21.838580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dab778-50b7-4aa9-bfff-05dabec31e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ee5b84d1a04a39a7bcc218c327d13a, user_id=d10980e9d3b94047aa81a8646c91c336, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:21.846550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dab778-50b7-4aa9-bfff-05dabec31e74 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:21.853348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2c27b-f52f-4702-9c4e-be995dbaf1c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd6cd4d3ca7423ca5490797475264f4, user_id=021d7fdc72b849668ea3765c5767c109, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:21.869206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2c27b-f52f-4702-9c4e-be995dbaf1c5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:21.895899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 895/1801] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:21 2022] PUT /identity/v3/projects/36ee5b84d1a04a39a7bcc218c327d13a/users/d10980e9d3b94047aa81a8646c91c336/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:21.897097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dab778-50b7-4aa9-bfff-05dabec31e74 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:21.928540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 907/1802] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:21 2022] PUT /identity/v3/projects/9fd6cd4d3ca7423ca5490797475264f4/users/021d7fdc72b849668ea3765c5767c109/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:21.931107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85101242-ce69-4770-b0b5-83e7453786c2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:21.931396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85101242-ce69-4770-b0b5-83e7453786c2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:21.931674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85101242-ce69-4770-b0b5-83e7453786c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:22.019113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-85101242-ce69-4770-b0b5-83e7453786c2 None None] MFA Rules not processed for user `d10980e9d3b94047aa81a8646c91c336`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:22.023728 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-85101242-ce69-4770-b0b5-83e7453786c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:22.142144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dab778-50b7-4aa9-bfff-05dabec31e74 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:22.250885 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 908/1803] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:21 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:22.252331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85101242-ce69-4770-b0b5-83e7453786c2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:22.284934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16318925-4ad2-4b51-841f-06b4cb4bc70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:22.286146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16318925-4ad2-4b51-841f-06b4cb4bc70f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:22.286542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16318925-4ad2-4b51-841f-06b4cb4bc70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:22.286922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16318925-4ad2-4b51-841f-06b4cb4bc70f admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:22.287912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16318925-4ad2-4b51-841f-06b4cb4bc70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:22.289945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16318925-4ad2-4b51-841f-06b4cb4bc70f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:22.306299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 896/1804] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:22.310681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:22.312907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16318925-4ad2-4b51-841f-06b4cb4bc70f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:22.317405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-16318925-4ad2-4b51-841f-06b4cb4bc70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:22.363002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16318925-4ad2-4b51-841f-06b4cb4bc70f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:22.378581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:22.379810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:22.380253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:22.380757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] PATH_INFO: `/v3/users/4fbf0c15ce10455ba894fbb225f25b40` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:22.382187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fbf0c15ce10455ba894fbb225f25b40)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:22.384477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:22.435809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85101242-ce69-4770-b0b5-83e7453786c2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:22.447767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df928c09-8784-4c22-89c2-085368c4c9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:22.449438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df928c09-8784-4c22-89c2-085368c4c9fa admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:22.450352 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df928c09-8784-4c22-89c2-085368c4c9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:22.450648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df928c09-8784-4c22-89c2-085368c4c9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:22.452568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df928c09-8784-4c22-89c2-085368c4c9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:22.457482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df928c09-8784-4c22-89c2-085368c4c9fa admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:22.463969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:22.484328 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 909/1805] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:22.484328 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df928c09-8784-4c22-89c2-085368c4c9fa admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:22.499180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fbf0c15ce10455ba894fbb225f25b40'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:22.500926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fbf0c15ce10455ba894fbb225f25b40'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:22.501624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fbf0c15ce10455ba894fbb225f25b40'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:22.524935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fbf0c15ce10455ba894fbb225f25b40'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:22.578699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fbf0c15ce10455ba894fbb225f25b40'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:22.612269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df928c09-8784-4c22-89c2-085368c4c9fa admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:22.619402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 897/1806] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:22 2022] DELETE /identity/v3/users/4fbf0c15ce10455ba894fbb225f25b40 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:22.619402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:22.619651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00ea98-e1fc-4f74-9743-4948180b70e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:22.619651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00ea98-e1fc-4f74-9743-4948180b70e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:22.619651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00ea98-e1fc-4f74-9743-4948180b70e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:22.619862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:22.627147 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00ea98-e1fc-4f74-9743-4948180b70e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:22.629890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8b00ea98-e1fc-4f74-9743-4948180b70e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:22.662308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00ea98-e1fc-4f74-9743-4948180b70e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:22.663925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00ea98-e1fc-4f74-9743-4948180b70e8 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:22.704983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 910/1807] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:22 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:22.706714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800508b-f246-4134-bc9e-275663597d85 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:22.708717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800508b-f246-4134-bc9e-275663597d85 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:22.709495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800508b-f246-4134-bc9e-275663597d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:22.785484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c800508b-f246-4134-bc9e-275663597d85 None None] MFA Rules not processed for user `1bf6e7a4afaf405caae7d750973d9956`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:22.786886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c800508b-f246-4134-bc9e-275663597d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:22.803926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e138-4e4f-4b0b-81e8-f6a211432723 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:22.832633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9e19b-fe63-438c-859c-2607abddbaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:22.834895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9e19b-fe63-438c-859c-2607abddbaa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:22.835188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9e19b-fe63-438c-859c-2607abddbaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:22.835471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9e19b-fe63-438c-859c-2607abddbaa5 admin admin] PATH_INFO: `/v3/projects/e8ab6188ef1c4297b52720d0359b6ae8/users/49522fcc32564bdbbbefd20dc2b388a6/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:22.843284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9e19b-fe63-438c-859c-2607abddbaa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8ab6188ef1c4297b52720d0359b6ae8, user_id=49522fcc32564bdbbbefd20dc2b388a6, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:22.849347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9e19b-fe63-438c-859c-2607abddbaa5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:22.916289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 898/1808] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:22 2022] PUT /identity/v3/projects/e8ab6188ef1c4297b52720d0359b6ae8/users/49522fcc32564bdbbbefd20dc2b388a6/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:22.916289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9e19b-fe63-438c-859c-2607abddbaa5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:22.916289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c1e9e19b-fe63-438c-859c-2607abddbaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:23.009563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 911/1809] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:23.011565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d6eaa-f623-4934-8649-13fc01fbb1a8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:23.011862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d6eaa-f623-4934-8649-13fc01fbb1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:23.012316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d6eaa-f623-4934-8649-13fc01fbb1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:23.049829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8f1d6eaa-f623-4934-8649-13fc01fbb1a8 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:23.051094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8f1d6eaa-f623-4934-8649-13fc01fbb1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:23.110946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9e19b-fe63-438c-859c-2607abddbaa5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:23.135432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67922a2-4814-454c-b7c0-5d324b83707c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:23.137142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67922a2-4814-454c-b7c0-5d324b83707c admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:23.137497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67922a2-4814-454c-b7c0-5d324b83707c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:23.137823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67922a2-4814-454c-b7c0-5d324b83707c admin admin] PATH_INFO: `/v3/projects/9fd6cd4d3ca7423ca5490797475264f4/users/021d7fdc72b849668ea3765c5767c109/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:23.141540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67922a2-4814-454c-b7c0-5d324b83707c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd6cd4d3ca7423ca5490797475264f4, user_id=021d7fdc72b849668ea3765c5767c109, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:23.147827 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67922a2-4814-454c-b7c0-5d324b83707c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:23.206511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 899/1810] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:22 2022] PUT /identity/v3/projects/9fd6cd4d3ca7423ca5490797475264f4/users/021d7fdc72b849668ea3765c5767c109/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:23.211343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2b44d-bde9-405f-ae87-14c279ac124c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:23.211635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2b44d-bde9-405f-ae87-14c279ac124c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:23.211917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2b44d-bde9-405f-ae87-14c279ac124c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:23.240617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 912/1811] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:23.243114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10ca30-0e59-4e6e-b39b-6a377f4d20d7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:23.243410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10ca30-0e59-4e6e-b39b-6a377f4d20d7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:23.243689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10ca30-0e59-4e6e-b39b-6a377f4d20d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:23.262881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:23.320403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8b10ca30-0e59-4e6e-b39b-6a377f4d20d7 None None] MFA Rules not processed for user `49522fcc32564bdbbbefd20dc2b388a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:23.324558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8b10ca30-0e59-4e6e-b39b-6a377f4d20d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:23.394602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:23.405667 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0fd2b44d-bde9-405f-ae87-14c279ac124c None None] MFA Rules not processed for user `75f11eaa4b504d0aa4018fb4ffbbd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:23.406985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0fd2b44d-bde9-405f-ae87-14c279ac124c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:23.475709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 913/1812] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:23.475709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacfcee-dd0f-4c57-b351-8a5ea790c4a5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:23.475709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacfcee-dd0f-4c57-b351-8a5ea790c4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:23.475709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacfcee-dd0f-4c57-b351-8a5ea790c4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:23.509050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9cacfcee-dd0f-4c57-b351-8a5ea790c4a5 None None] MFA Rules not processed for user `021d7fdc72b849668ea3765c5767c109`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:23.510896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9cacfcee-dd0f-4c57-b351-8a5ea790c4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:23.622142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 900/1813] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 17:34:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:23.623439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2b44d-bde9-405f-ae87-14c279ac124c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:23.631654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0fd2b44d-bde9-405f-ae87-14c279ac124c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:23.696243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 914/1814] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:23.696243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacfcee-dd0f-4c57-b351-8a5ea790c4a5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:23.698667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9cacfcee-dd0f-4c57-b351-8a5ea790c4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:23.787686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2b44d-bde9-405f-ae87-14c279ac124c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:23.798740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8bcc8-3cc2-4df0-9b1f-04c441caf9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:23.800475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8bcc8-3cc2-4df0-9b1f-04c441caf9ab service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:23.801080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8bcc8-3cc2-4df0-9b1f-04c441caf9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:23.801080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8bcc8-3cc2-4df0-9b1f-04c441caf9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:23.849068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacfcee-dd0f-4c57-b351-8a5ea790c4a5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:23.859217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f773bf-d9f0-4563-a450-786bb0c10533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:23.860000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f773bf-d9f0-4563-a450-786bb0c10533 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:23.860412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f773bf-d9f0-4563-a450-786bb0c10533 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:23.860920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f773bf-d9f0-4563-a450-786bb0c10533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:23.884040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f773bf-d9f0-4563-a450-786bb0c10533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:23.887417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f773bf-d9f0-4563-a450-786bb0c10533 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:23.909095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 915/1815] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:23.912135 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f773bf-d9f0-4563-a450-786bb0c10533 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:23.916243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-18f773bf-d9f0-4563-a450-786bb0c10533 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:23.939093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f773bf-d9f0-4563-a450-786bb0c10533 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:23.939845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8bcc8-3cc2-4df0-9b1f-04c441caf9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:23.943419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8bcc8-3cc2-4df0-9b1f-04c441caf9ab service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:23.952533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036cab4-c519-4370-8182-181ec4e408f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:23.953301 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036cab4-c519-4370-8182-181ec4e408f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:23.953654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036cab4-c519-4370-8182-181ec4e408f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:23.954000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036cab4-c519-4370-8182-181ec4e408f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:23.972205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 901/1816] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:23.975782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036cab4-c519-4370-8182-181ec4e408f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:23.977550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036cab4-c519-4370-8182-181ec4e408f1 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:23.981997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8bcc8-3cc2-4df0-9b1f-04c441caf9ab service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:23.984423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-40c8bcc8-3cc2-4df0-9b1f-04c441caf9ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:24.023412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 916/1817] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:24.065779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036cab4-c519-4370-8182-181ec4e408f1 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:24.068632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3036cab4-c519-4370-8182-181ec4e408f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:24.090760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:24.104109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036cab4-c519-4370-8182-181ec4e408f1 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:24.120892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:24.122354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:24.123842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:24.125195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] PATH_INFO: `/v3/users/5ac91a12c8fd427092042eced55fc61f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:24.129693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac91a12c8fd427092042eced55fc61f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:24.154313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:24.184406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:24.248088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8bcc8-3cc2-4df0-9b1f-04c441caf9ab service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:24.254889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad1347-521b-4df4-acab-1da43c263762 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a26412cf9f64b1bbbd8e595b55bcebc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:24.255930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad1347-521b-4df4-acab-1da43c263762 service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:24.256438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad1347-521b-4df4-acab-1da43c263762 service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:24.256974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad1347-521b-4df4-acab-1da43c263762 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:24.268628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad1347-521b-4df4-acab-1da43c263762 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:24.270975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad1347-521b-4df4-acab-1da43c263762 service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:24.295019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 902/1818] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jul 21 17:34:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 17:34:24.296327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad1347-521b-4df4-acab-1da43c263762 service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:24.298414 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac91a12c8fd427092042eced55fc61f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:24.311049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad1347-521b-4df4-acab-1da43c263762 service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:24.316439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:24.317402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:24.317758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:24.317926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac91a12c8fd427092042eced55fc61f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:24.318153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] PATH_INFO: `/v3/users/99ae98d641654d5ca21f50c8540981cf` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:24.319542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99ae98d641654d5ca21f50c8540981cf)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:24.320971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac91a12c8fd427092042eced55fc61f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:24.321675 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:24.348946 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac91a12c8fd427092042eced55fc61f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:24.366142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac91a12c8fd427092042eced55fc61f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:24.382838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 917/1819] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:24 2022] DELETE /identity/v3/users/5ac91a12c8fd427092042eced55fc61f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:24.384031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:24.389380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:24.441616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99ae98d641654d5ca21f50c8540981cf'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:24.442675 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99ae98d641654d5ca21f50c8540981cf'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:24.443489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99ae98d641654d5ca21f50c8540981cf'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:24.453032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99ae98d641654d5ca21f50c8540981cf'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:24.487852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5b01311c-b101-4d29-91c5-4cf91a542d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99ae98d641654d5ca21f50c8540981cf'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:24.550718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 903/1820] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:24 2022] DELETE /identity/v3/users/99ae98d641654d5ca21f50c8540981cf => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:24.552237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2606c6-3275-4706-86de-ad17f396f6a0 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:24.552806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2606c6-3275-4706-86de-ad17f396f6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:24.553495 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2606c6-3275-4706-86de-ad17f396f6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:24.575917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfb402-5ecc-4279-8ec2-84972b5aeefd admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:24.606295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0d2606c6-3275-4706-86de-ad17f396f6a0 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:24.607849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0d2606c6-3275-4706-86de-ad17f396f6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:24.723694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66757653-b893-4617-a569-58ec2c381e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:24.724826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66757653-b893-4617-a569-58ec2c381e82 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:24.725772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66757653-b893-4617-a569-58ec2c381e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:24.726266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66757653-b893-4617-a569-58ec2c381e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:24.734304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-66757653-b893-4617-a569-58ec2c381e82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:24.765989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 904/1821] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:24.769620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16c238-193d-403f-9420-08a9a9042ec8 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:24.769620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16c238-193d-403f-9420-08a9a9042ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:24.769620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16c238-193d-403f-9420-08a9a9042ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:24.803085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7f16c238-193d-403f-9420-08a9a9042ec8 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:24.804365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7f16c238-193d-403f-9420-08a9a9042ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:24.825421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 905/1822] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:24.832280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16c238-193d-403f-9420-08a9a9042ec8 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:24.846194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7f16c238-193d-403f-9420-08a9a9042ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:24.870797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:24.901104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66757653-b893-4617-a569-58ec2c381e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:24.903460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66757653-b893-4617-a569-58ec2c381e82 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:24.922939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:24.936837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 918/1823] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:24.940562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66757653-b893-4617-a569-58ec2c381e82 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:24.945863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-66757653-b893-4617-a569-58ec2c381e82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:24.985589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16c238-193d-403f-9420-08a9a9042ec8 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:24.996031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23815156-ae1f-40eb-9d25-2d4d058160e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:24.996786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23815156-ae1f-40eb-9d25-2d4d058160e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:24.997019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23815156-ae1f-40eb-9d25-2d4d058160e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:24.997299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23815156-ae1f-40eb-9d25-2d4d058160e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:25.023470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23815156-ae1f-40eb-9d25-2d4d058160e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:25.024974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23815156-ae1f-40eb-9d25-2d4d058160e0 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:25.059915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 906/1824] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:34:25.180878 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66757653-b893-4617-a569-58ec2c381e82 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:25.187316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda8014-94d7-49e6-a719-d720309f6e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:25.188996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda8014-94d7-49e6-a719-d720309f6e48 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:25.189158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda8014-94d7-49e6-a719-d720309f6e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:25.189558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda8014-94d7-49e6-a719-d720309f6e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:25.217601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda8014-94d7-49e6-a719-d720309f6e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:25.219229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda8014-94d7-49e6-a719-d720309f6e48 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:25.250829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 919/1825] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:25.826188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02c72c-8280-4782-96b8-fb6861714606 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:25.826188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02c72c-8280-4782-96b8-fb6861714606 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:25.826482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02c72c-8280-4782-96b8-fb6861714606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:25.876727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fb02c72c-8280-4782-96b8-fb6861714606 None None] MFA Rules not processed for user `b0edda384caf46819169b69b858afc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:25.881947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fb02c72c-8280-4782-96b8-fb6861714606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:25.909855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 907/1826] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jul 21 17:34:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:26.024880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda8014-94d7-49e6-a719-d720309f6e48 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:26.027378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9cda8014-94d7-49e6-a719-d720309f6e48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:26.045024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda8014-94d7-49e6-a719-d720309f6e48 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:26.053417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8448b-c19b-4ab0-9323-3e9ac0833e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:26.054289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8448b-c19b-4ab0-9323-3e9ac0833e1a service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:26.054674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8448b-c19b-4ab0-9323-3e9ac0833e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:26.055045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8448b-c19b-4ab0-9323-3e9ac0833e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:26.077964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8448b-c19b-4ab0-9323-3e9ac0833e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:26.079611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8448b-c19b-4ab0-9323-3e9ac0833e1a service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:26.105471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 920/1827] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:34:28.113316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02c72c-8280-4782-96b8-fb6861714606 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:28.116946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fb02c72c-8280-4782-96b8-fb6861714606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:28.170347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02c72c-8280-4782-96b8-fb6861714606 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:28.177045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:28.178182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:28.178739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:28.179258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] PATH_INFO: `/v3/projects/29cbddf77dd9434caf011384f338bfcf` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:28.183651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29cbddf77dd9434caf011384f338bfcf)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:28.185890 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:28.358493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29cbddf77dd9434caf011384f338bfcf', 'request_id': 'req-9d944615-c56d-4e88-b487-71c67a5fa366'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:28.375595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29cbddf77dd9434caf011384f338bfcf', 'request_id': 'req-9d944615-c56d-4e88-b487-71c67a5fa366'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:28.390750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29cbddf77dd9434caf011384f338bfcf', 'request_id': 'req-9d944615-c56d-4e88-b487-71c67a5fa366'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:28.416199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] The token cache is being invalidate because project 29cbddf77dd9434caf011384f338bfcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:28.419641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:28.421313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:28.424080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 908/1828] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:28 2022] DELETE /identity/v3/projects/29cbddf77dd9434caf011384f338bfcf => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:28.428013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8448b-c19b-4ab0-9323-3e9ac0833e1a service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:28.453555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8448b-c19b-4ab0-9323-3e9ac0833e1a service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:28.615219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:28.617747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:28.619479 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:28.620710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] PATH_INFO: `/v3/users/49522fcc32564bdbbbefd20dc2b388a6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:28.624220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49522fcc32564bdbbbefd20dc2b388a6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:28.628102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:28.629589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:28.630132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:28.663908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d944615-c56d-4e88-b487-71c67a5fa366 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:28.680099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:28.680099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:28.680383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:28.680734 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] PATH_INFO: `/v3/projects/e02d044094384d47bb23a2a9580e3b54` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:28.685165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02d044094384d47bb23a2a9580e3b54)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:28.688492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:28.817205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49522fcc32564bdbbbefd20dc2b388a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:28.858421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49522fcc32564bdbbbefd20dc2b388a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:28.860504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49522fcc32564bdbbbefd20dc2b388a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:28.953170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49522fcc32564bdbbbefd20dc2b388a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:28.977366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02d044094384d47bb23a2a9580e3b54', 'request_id': 'req-59b8a0b5-173e-413d-a7c0-2546d20359ee'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:28.981315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:28.994814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01c70d2c-8c75-4e25-9c6a-a31c8ac85bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49522fcc32564bdbbbefd20dc2b388a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.005234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02d044094384d47bb23a2a9580e3b54', 'request_id': 'req-59b8a0b5-173e-413d-a7c0-2546d20359ee'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.024916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02d044094384d47bb23a2a9580e3b54', 'request_id': 'req-59b8a0b5-173e-413d-a7c0-2546d20359ee'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.045828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 921/1829] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:28 2022] DELETE /identity/v3/users/49522fcc32564bdbbbefd20dc2b388a6 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:29.057317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] The token cache is being invalidate because project e02d044094384d47bb23a2a9580e3b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:29.059676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.060047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc1f20-1913-4095-bec0-cca7b5df23a8 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:29.062223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc1f20-1913-4095-bec0-cca7b5df23a8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:29.062760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc1f20-1913-4095-bec0-cca7b5df23a8 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:29.063865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.064679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 922/1830] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 17:34:29 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:34:29.074432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc1f20-1913-4095-bec0-cca7b5df23a8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:29.074722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 909/1831] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:28 2022] DELETE /identity/v3/projects/e02d044094384d47bb23a2a9580e3b54 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:29.075967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1ddc1f20-1913-4095-bec0-cca7b5df23a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:29.077008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:29.088204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:29.322842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:29.411784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8a0b5-173e-413d-a7c0-2546d20359ee admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:29.419083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc1f20-1913-4095-bec0-cca7b5df23a8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:29.426891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:29.429869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:29.430162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:29.430962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] PATH_INFO: `/v3/users/d847298e58234177bd921c7c78715ae7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:29.431529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:29.434837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d847298e58234177bd921c7c78715ae7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:29.435368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:29.435665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:29.436109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] PATH_INFO: `/v3/users/9d8ea15803c647dca00295f3979ecb53` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:29.439353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d8ea15803c647dca00295f3979ecb53)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:29.440695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:29.447793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:29.589238 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd847298e58234177bd921c7c78715ae7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.590183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd847298e58234177bd921c7c78715ae7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.591184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd847298e58234177bd921c7c78715ae7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.609700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd847298e58234177bd921c7c78715ae7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.648527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d8ea15803c647dca00295f3979ecb53'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.665688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b0becda9-991e-4617-b88c-856276feece9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd847298e58234177bd921c7c78715ae7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.670308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d8ea15803c647dca00295f3979ecb53'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.676069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d8ea15803c647dca00295f3979ecb53'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.717307 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 910/1832] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:29 2022] DELETE /identity/v3/users/d847298e58234177bd921c7c78715ae7 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:29.721868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058eb76-be26-4393-84a4-0cb113619aa6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:29.729764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d8ea15803c647dca00295f3979ecb53'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.730447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058eb76-be26-4393-84a4-0cb113619aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:29.731977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058eb76-be26-4393-84a4-0cb113619aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:29.750714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d8ea15803c647dca00295f3979ecb53'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:29.765905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 923/1833] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:29 2022] DELETE /identity/v3/users/9d8ea15803c647dca00295f3979ecb53 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:29.768322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:29.780627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:29.845861 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3058eb76-be26-4393-84a4-0cb113619aa6 None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:29.853188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3058eb76-be26-4393-84a4-0cb113619aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:30.017099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 911/1834] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 17:34:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:30.018413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324395d-4e79-4fb9-bf38-e07138bdef24 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:30.018897 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324395d-4e79-4fb9-bf38-e07138bdef24 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:30.019288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324395d-4e79-4fb9-bf38-e07138bdef24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:30.040351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2324395d-4e79-4fb9-bf38-e07138bdef24 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:30.041826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2324395d-4e79-4fb9-bf38-e07138bdef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:30.061642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc1914-fabe-4037-a4bc-a409ac97c9af admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:30.070247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 912/1835] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:30.083583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:30.085659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:30.086424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:30.087019 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] PATH_INFO: `/v3/projects/cf2f3b226528454bba5168b0a972f4d0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:30.097819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf2f3b226528454bba5168b0a972f4d0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:30.106637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:30.127530 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324395d-4e79-4fb9-bf38-e07138bdef24 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:30.130611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2324395d-4e79-4fb9-bf38-e07138bdef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:30.175456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:30.210893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:30.267162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324395d-4e79-4fb9-bf38-e07138bdef24 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:30.311144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf2f3b226528454bba5168b0a972f4d0', 'request_id': 'req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:30.321487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf2f3b226528454bba5168b0a972f4d0', 'request_id': 'req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:30.339435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf2f3b226528454bba5168b0a972f4d0', 'request_id': 'req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:30.358353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] The token cache is being invalidate because project cf2f3b226528454bba5168b0a972f4d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:30.358632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:30.359738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:30.361783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 924/1836] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:29 2022] DELETE /identity/v3/projects/cf2f3b226528454bba5168b0a972f4d0 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:30.365840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:30.367301 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:30.407775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab955d7-043c-4b38-984e-b96b431002ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:30.408503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab955d7-043c-4b38-984e-b96b431002ea service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:30.408938 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab955d7-043c-4b38-984e-b96b431002ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:30.409228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab955d7-043c-4b38-984e-b96b431002ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:30.412973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aab955d7-043c-4b38-984e-b96b431002ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:30.462916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf66ee1-9e7d-404d-98fd-19704ad9f6df admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:30.470399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:30.471844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:30.472285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:30.472727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] PATH_INFO: `/v3/users/d10980e9d3b94047aa81a8646c91c336` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:30.474183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] RBAC: Authorizing `identity:delete_user(user_id=d10980e9d3b94047aa81a8646c91c336)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:30.476964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:30.548499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab955d7-043c-4b38-984e-b96b431002ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:30.556639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab955d7-043c-4b38-984e-b96b431002ea service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:30.556762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd10980e9d3b94047aa81a8646c91c336'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:30.587527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd10980e9d3b94047aa81a8646c91c336'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:30.590564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd10980e9d3b94047aa81a8646c91c336'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:30.605650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 913/1837] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:34:30.639776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd10980e9d3b94047aa81a8646c91c336'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:30.662116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-965d2a3b-92df-453d-9193-408a42e28def admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd10980e9d3b94047aa81a8646c91c336'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:30.675888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 925/1838] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:30 2022] DELETE /identity/v3/users/d10980e9d3b94047aa81a8646c91c336 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:31.041433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de37ead-2bec-4410-9256-1b00958eec31 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:31.041908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de37ead-2bec-4410-9256-1b00958eec31 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:31.042865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de37ead-2bec-4410-9256-1b00958eec31 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:31.045090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 914/1839] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 17:34:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:34:31.122939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a632ee7-c024-47f6-8131-419b0d5f2184 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:31.123217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a632ee7-c024-47f6-8131-419b0d5f2184 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:31.123496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a632ee7-c024-47f6-8131-419b0d5f2184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:31.127917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de37ead-2bec-4410-9256-1b00958eec31 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:31.131476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4de37ead-2bec-4410-9256-1b00958eec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:31.182274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7a632ee7-c024-47f6-8131-419b0d5f2184 None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:31.185746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7a632ee7-c024-47f6-8131-419b0d5f2184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:31.275250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:31.362037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de37ead-2bec-4410-9256-1b00958eec31 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:31.373515 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60372c8b-312a-4022-b0aa-bdaaeaac7e21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:31.374349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60372c8b-312a-4022-b0aa-bdaaeaac7e21 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:31.374922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60372c8b-312a-4022-b0aa-bdaaeaac7e21 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:31.375100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60372c8b-312a-4022-b0aa-bdaaeaac7e21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:31.381184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-60372c8b-312a-4022-b0aa-bdaaeaac7e21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:31.408876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60372c8b-312a-4022-b0aa-bdaaeaac7e21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:31.410790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60372c8b-312a-4022-b0aa-bdaaeaac7e21 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:31.447801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 915/1840] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:31.452217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 926/1841] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 17:34:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:31.562172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60372c8b-312a-4022-b0aa-bdaaeaac7e21 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:31.578376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60372c8b-312a-4022-b0aa-bdaaeaac7e21 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:31.600836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97def9b0-8b31-4865-9db5-ac43be603cc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:31.601931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def9b0-8b31-4865-9db5-ac43be603cc5 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:31.601931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def9b0-8b31-4865-9db5-ac43be603cc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:31.602186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def9b0-8b31-4865-9db5-ac43be603cc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:31.625157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97def9b0-8b31-4865-9db5-ac43be603cc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:31.626838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97def9b0-8b31-4865-9db5-ac43be603cc5 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:31.645693 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 916/1842] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:32.039953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a632ee7-c024-47f6-8131-419b0d5f2184 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:32.041510 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7a632ee7-c024-47f6-8131-419b0d5f2184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:32.225699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a632ee7-c024-47f6-8131-419b0d5f2184 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:32.231467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:32.232538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:32.232940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:32.233325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] PATH_INFO: `/v3/projects/e8ab6188ef1c4297b52720d0359b6ae8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:32.234664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8ab6188ef1c4297b52720d0359b6ae8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:32.244459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:32.390544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8ab6188ef1c4297b52720d0359b6ae8', 'request_id': 'req-c01270dd-2470-4148-bbd9-613401fb23f5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:32.405284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8ab6188ef1c4297b52720d0359b6ae8', 'request_id': 'req-c01270dd-2470-4148-bbd9-613401fb23f5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:32.440679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8ab6188ef1c4297b52720d0359b6ae8', 'request_id': 'req-c01270dd-2470-4148-bbd9-613401fb23f5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:32.464338 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] The token cache is being invalidate because project e8ab6188ef1c4297b52720d0359b6ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:32.464616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:32.465305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c01270dd-2470-4148-bbd9-613401fb23f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:32.477186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 927/1843] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:32 2022] DELETE /identity/v3/projects/e8ab6188ef1c4297b52720d0359b6ae8 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:32.484887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f401cc-e257-4552-b39d-66a5ea59903e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:32.486113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f401cc-e257-4552-b39d-66a5ea59903e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:32.486504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f401cc-e257-4552-b39d-66a5ea59903e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:32.622434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-95f401cc-e257-4552-b39d-66a5ea59903e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:32.623929 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-95f401cc-e257-4552-b39d-66a5ea59903e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:32.650237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:32.723831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 917/1844] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:32.734061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca872c-d13e-4cba-95e8-01fd5055738e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:32.734521 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca872c-d13e-4cba-95e8-01fd5055738e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:32.735291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca872c-d13e-4cba-95e8-01fd5055738e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:32.757821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b8ca872c-d13e-4cba-95e8-01fd5055738e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:32.759304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b8ca872c-d13e-4cba-95e8-01fd5055738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:32.770403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 928/1845] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:32.774429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f401cc-e257-4552-b39d-66a5ea59903e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:32.793288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f401cc-e257-4552-b39d-66a5ea59903e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:32.804669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7f9fd-a15c-447c-b960-d93d4998888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:32.808554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7f9fd-a15c-447c-b960-d93d4998888c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:32.809559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7f9fd-a15c-447c-b960-d93d4998888c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:32.811554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7f9fd-a15c-447c-b960-d93d4998888c admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:32.813032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7f9fd-a15c-447c-b960-d93d4998888c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:32.813863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7f9fd-a15c-447c-b960-d93d4998888c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:32.818212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7f9fd-a15c-447c-b960-d93d4998888c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:32.847822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca872c-d13e-4cba-95e8-01fd5055738e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:32.848526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 918/1846] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:32.858913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca872c-d13e-4cba-95e8-01fd5055738e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:32.874179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d61ced-8b9d-45bd-aaca-b091fbb98fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:32.874179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61ced-8b9d-45bd-aaca-b091fbb98fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:32.874179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61ced-8b9d-45bd-aaca-b091fbb98fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:32.874179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61ced-8b9d-45bd-aaca-b091fbb98fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:32.876420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d61ced-8b9d-45bd-aaca-b091fbb98fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:32.895443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d61ced-8b9d-45bd-aaca-b091fbb98fa8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:32.958032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7f9fd-a15c-447c-b960-d93d4998888c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:32.970767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 929/1847] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:32 2022] POST /identity/v3/projects => generated 399 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:32.973283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7f9fd-a15c-447c-b960-d93d4998888c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:33.151352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd418f73-174c-406c-a82b-82d0ff8fb430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:33.151635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd418f73-174c-406c-a82b-82d0ff8fb430 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:33.152896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd418f73-174c-406c-a82b-82d0ff8fb430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:33.152896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd418f73-174c-406c-a82b-82d0ff8fb430 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:33.153250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd418f73-174c-406c-a82b-82d0ff8fb430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:33.156084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd418f73-174c-406c-a82b-82d0ff8fb430 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:33.193287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 919/1848] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:32 2022] POST /identity/v3/users => generated 300 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:33.197144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d61ced-8b9d-45bd-aaca-b091fbb98fa8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:33.227961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d61ced-8b9d-45bd-aaca-b091fbb98fa8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:33.241897 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ab6d9-e10c-497b-8cb7-7bb7287f0af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:33.242693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ab6d9-e10c-497b-8cb7-7bb7287f0af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:33.242977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ab6d9-e10c-497b-8cb7-7bb7287f0af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:33.243318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ab6d9-e10c-497b-8cb7-7bb7287f0af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:33.244070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ab6d9-e10c-497b-8cb7-7bb7287f0af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:33.246844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ab6d9-e10c-497b-8cb7-7bb7287f0af8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:33.279722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 930/1849] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:33.280845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd418f73-174c-406c-a82b-82d0ff8fb430 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:33.299864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd418f73-174c-406c-a82b-82d0ff8fb430 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:33.318467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d603432-0727-46e7-85c3-b8bc2863b651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:33.319262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603432-0727-46e7-85c3-b8bc2863b651 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:33.319644 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603432-0727-46e7-85c3-b8bc2863b651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:33.319929 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603432-0727-46e7-85c3-b8bc2863b651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:33.320793 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d603432-0727-46e7-85c3-b8bc2863b651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:33.322763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d603432-0727-46e7-85c3-b8bc2863b651 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:33.355497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 920/1850] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:33.359559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ab6d9-e10c-497b-8cb7-7bb7287f0af8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:33.377743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ab6d9-e10c-497b-8cb7-7bb7287f0af8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:33.388848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d683cb-3878-45b4-bad3-af0861384e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:33.390046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d683cb-3878-45b4-bad3-af0861384e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:33.390688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d683cb-3878-45b4-bad3-af0861384e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:33.390797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d683cb-3878-45b4-bad3-af0861384e96 admin admin] PATH_INFO: `/v3/projects/932b6c44526b445fa0977d5614687bf2/users/db532044d18b4370b72b59757909b789/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:33.401187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:33.431501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d683cb-3878-45b4-bad3-af0861384e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=932b6c44526b445fa0977d5614687bf2, user_id=db532044d18b4370b72b59757909b789, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:33.435448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d683cb-3878-45b4-bad3-af0861384e96 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:33.478144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 931/1851] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:33 2022] PUT /identity/v3/projects/932b6c44526b445fa0977d5614687bf2/users/db532044d18b4370b72b59757909b789/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:33.492875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cef12-2b4a-4b0d-891f-045451de4518 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:33.493492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cef12-2b4a-4b0d-891f-045451de4518 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:33.493492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cef12-2b4a-4b0d-891f-045451de4518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:33.556324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d683cb-3878-45b4-bad3-af0861384e96 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:33.558591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c7d683cb-3878-45b4-bad3-af0861384e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:33.604949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d13cef12-2b4a-4b0d-891f-045451de4518 None None] MFA Rules not processed for user `db532044d18b4370b72b59757909b789`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:33.610547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d13cef12-2b4a-4b0d-891f-045451de4518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:33.766043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d683cb-3878-45b4-bad3-af0861384e96 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:33.780930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 921/1852] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:33.787006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cef12-2b4a-4b0d-891f-045451de4518 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:33.790181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:33.791353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:33.792283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:33.792883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] PATH_INFO: `/v3/projects/36ee5b84d1a04a39a7bcc218c327d13a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:33.798127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ee5b84d1a04a39a7bcc218c327d13a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:33.802144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:33.825120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cef12-2b4a-4b0d-891f-045451de4518 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:33.838043 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602bb38-df1c-4c77-85a6-2108c6ca53b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:33.840201 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602bb38-df1c-4c77-85a6-2108c6ca53b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:33.841097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602bb38-df1c-4c77-85a6-2108c6ca53b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:33.842358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602bb38-df1c-4c77-85a6-2108c6ca53b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:33.843572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602bb38-df1c-4c77-85a6-2108c6ca53b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:33.849532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602bb38-df1c-4c77-85a6-2108c6ca53b2 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:33.864137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:33.955473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 922/1853] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:33 2022] POST /identity/v3/projects => generated 401 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:34:33.957368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602bb38-df1c-4c77-85a6-2108c6ca53b2 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:33.975154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ee5b84d1a04a39a7bcc218c327d13a', 'request_id': 'req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:33.995981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ee5b84d1a04a39a7bcc218c327d13a', 'request_id': 'req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:34.022513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ee5b84d1a04a39a7bcc218c327d13a', 'request_id': 'req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:34.053298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] The token cache is being invalidate because project 36ee5b84d1a04a39a7bcc218c327d13a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:34.053800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:34.056953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7f0d4d1b-3963-49b4-916f-9960d75c3b3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:34.072338 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 932/1854] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:33 2022] DELETE /identity/v3/projects/36ee5b84d1a04a39a7bcc218c327d13a => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:34.078768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b744-9351-488d-8f41-bbf89658dc37 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.079286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b744-9351-488d-8f41-bbf89658dc37 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.079722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b744-9351-488d-8f41-bbf89658dc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.125614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4996b744-9351-488d-8f41-bbf89658dc37 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:34.132098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4996b744-9351-488d-8f41-bbf89658dc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:34.149373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602bb38-df1c-4c77-85a6-2108c6ca53b2 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:34.160200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 933/1855] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:34.161536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6115a-641b-4e1d-975b-408fb234a419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:34.162248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6115a-641b-4e1d-975b-408fb234a419 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.162541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6115a-641b-4e1d-975b-408fb234a419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.162825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6115a-641b-4e1d-975b-408fb234a419 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.163539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6115a-641b-4e1d-975b-408fb234a419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:34.165647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6115a-641b-4e1d-975b-408fb234a419 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:34.173797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba5bf9-e7f9-4cc8-a0be-192d013e7746 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.174726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba5bf9-e7f9-4cc8-a0be-192d013e7746 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.175184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba5bf9-e7f9-4cc8-a0be-192d013e7746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.222893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 923/1856] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:33 2022] POST /identity/v3/users => generated 307 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:34.233670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0cba5bf9-e7f9-4cc8-a0be-192d013e7746 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:34.234935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0cba5bf9-e7f9-4cc8-a0be-192d013e7746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:34.242178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6115a-641b-4e1d-975b-408fb234a419 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:34.245443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-00e6115a-641b-4e1d-975b-408fb234a419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:34.267020 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6115a-641b-4e1d-975b-408fb234a419 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:34.271801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 934/1857] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:34.275896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba5bf9-e7f9-4cc8-a0be-192d013e7746 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:34.284315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a114b-6222-4fc3-a779-131026f15fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:34.287605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a114b-6222-4fc3-a779-131026f15fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.289490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a114b-6222-4fc3-a779-131026f15fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.290450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a114b-6222-4fc3-a779-131026f15fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.294368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a114b-6222-4fc3-a779-131026f15fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:34.298723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a114b-6222-4fc3-a779-131026f15fa5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:34.306996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba5bf9-e7f9-4cc8-a0be-192d013e7746 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:34.325974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85200535-3a37-47a7-b7b7-aa4146847169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:34.326677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85200535-3a37-47a7-b7b7-aa4146847169 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.326964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85200535-3a37-47a7-b7b7-aa4146847169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.327243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85200535-3a37-47a7-b7b7-aa4146847169 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.327989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85200535-3a37-47a7-b7b7-aa4146847169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:34.328320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85200535-3a37-47a7-b7b7-aa4146847169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:34.330791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85200535-3a37-47a7-b7b7-aa4146847169 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:34.335386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 924/1858] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:34.336989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a114b-6222-4fc3-a779-131026f15fa5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:34.356179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 935/1859] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:34.357517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85200535-3a37-47a7-b7b7-aa4146847169 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:34.365649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a114b-6222-4fc3-a779-131026f15fa5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:34.378415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85200535-3a37-47a7-b7b7-aa4146847169 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:34.383901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ccc1c-4a51-4cd1-b5df-4adab957f69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:34.385062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ccc1c-4a51-4cd1-b5df-4adab957f69c admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.385583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ccc1c-4a51-4cd1-b5df-4adab957f69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.385928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ccc1c-4a51-4cd1-b5df-4adab957f69c admin admin] PATH_INFO: `/v3/projects/66f40c9370d34e99aee8460199846b36/users/0b5d219b5998469d93dbe6d8394efb91/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.386660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d6600-d0ff-44e2-a5f1-52bfb7042595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:34.387686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d6600-d0ff-44e2-a5f1-52bfb7042595 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.388173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d6600-d0ff-44e2-a5f1-52bfb7042595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.389118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d6600-d0ff-44e2-a5f1-52bfb7042595 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.391609 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d6600-d0ff-44e2-a5f1-52bfb7042595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:34.396915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d6600-d0ff-44e2-a5f1-52bfb7042595 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:34.424874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ccc1c-4a51-4cd1-b5df-4adab957f69c admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f40c9370d34e99aee8460199846b36, user_id=0b5d219b5998469d93dbe6d8394efb91, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:34.430953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ccc1c-4a51-4cd1-b5df-4adab957f69c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:34.458689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 925/1860] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:34 2022] PUT /identity/v3/projects/66f40c9370d34e99aee8460199846b36/users/0b5d219b5998469d93dbe6d8394efb91/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:34.461695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 936/1861] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:34 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:34.462808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ccc1c-4a51-4cd1-b5df-4adab957f69c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:34.466677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d6600-d0ff-44e2-a5f1-52bfb7042595 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:34.597982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:34.619276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ccc1c-4a51-4cd1-b5df-4adab957f69c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:34.717085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d6600-d0ff-44e2-a5f1-52bfb7042595 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:34.730684 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:34.739828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2b61e-3d5e-4efc-bfb5-3a0c24290977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:34.742185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2b61e-3d5e-4efc-bfb5-3a0c24290977 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.742185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2b61e-3d5e-4efc-bfb5-3a0c24290977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.743523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2b61e-3d5e-4efc-bfb5-3a0c24290977 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.744463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2b61e-3d5e-4efc-bfb5-3a0c24290977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:34.746819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2b61e-3d5e-4efc-bfb5-3a0c24290977 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:34.755328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46bd87-a3c2-4bff-af7b-89d94c107331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:34.756077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46bd87-a3c2-4bff-af7b-89d94c107331 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.758837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46bd87-a3c2-4bff-af7b-89d94c107331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.759722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46bd87-a3c2-4bff-af7b-89d94c107331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.763807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46bd87-a3c2-4bff-af7b-89d94c107331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:34.768524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46bd87-a3c2-4bff-af7b-89d94c107331 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:34.806842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 937/1862] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:34 2022] POST /identity/v3/users => generated 283 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:34.808094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2b61e-3d5e-4efc-bfb5-3a0c24290977 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:34.811665 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 926/1863] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:34.812845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46bd87-a3c2-4bff-af7b-89d94c107331 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:34.813386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1ae2b61e-3d5e-4efc-bfb5-3a0c24290977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:34.830517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46bd87-a3c2-4bff-af7b-89d94c107331 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:34.838357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2b61e-3d5e-4efc-bfb5-3a0c24290977 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:34.853594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54051f1d-5e09-4ae1-9302-bb21f5e70c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:34.854520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54051f1d-5e09-4ae1-9302-bb21f5e70c92 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.854813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54051f1d-5e09-4ae1-9302-bb21f5e70c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.855097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54051f1d-5e09-4ae1-9302-bb21f5e70c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.855862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54051f1d-5e09-4ae1-9302-bb21f5e70c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:34.856751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:34.857607 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54051f1d-5e09-4ae1-9302-bb21f5e70c92 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:34.858106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.858616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.859112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] PATH_INFO: `/v3/projects/f5edf868e94648bb91a11134ddd09b52` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.862071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5edf868e94648bb91a11134ddd09b52)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:34.867809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:34.894807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 927/1864] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:34.895758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54051f1d-5e09-4ae1-9302-bb21f5e70c92 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:34.927174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54051f1d-5e09-4ae1-9302-bb21f5e70c92 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:34.941547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0615d06-d651-4df7-a34e-11bc0f13692e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:34.948724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0615d06-d651-4df7-a34e-11bc0f13692e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:34.949030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0615d06-d651-4df7-a34e-11bc0f13692e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:34.949347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0615d06-d651-4df7-a34e-11bc0f13692e admin admin] PATH_INFO: `/v3/projects/66f40c9370d34e99aee8460199846b36/users/0b5d219b5998469d93dbe6d8394efb91/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:34.962033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0615d06-d651-4df7-a34e-11bc0f13692e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f40c9370d34e99aee8460199846b36, user_id=0b5d219b5998469d93dbe6d8394efb91, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:34.969005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0615d06-d651-4df7-a34e-11bc0f13692e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:34.989840 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 928/1865] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:34 2022] PUT /identity/v3/projects/66f40c9370d34e99aee8460199846b36/users/0b5d219b5998469d93dbe6d8394efb91/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:34.991021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0615d06-d651-4df7-a34e-11bc0f13692e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:35.032979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5edf868e94648bb91a11134ddd09b52', 'request_id': 'req-7e953ba9-6d47-4aac-abbb-981f739b57aa'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.056835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5edf868e94648bb91a11134ddd09b52', 'request_id': 'req-7e953ba9-6d47-4aac-abbb-981f739b57aa'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.090508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5edf868e94648bb91a11134ddd09b52', 'request_id': 'req-7e953ba9-6d47-4aac-abbb-981f739b57aa'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.127351 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] The token cache is being invalidate because project f5edf868e94648bb91a11134ddd09b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:35.127678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.131735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.142628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 938/1866] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:34 2022] DELETE /identity/v3/projects/f5edf868e94648bb91a11134ddd09b52 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:35.145151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:35.149830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:35.212345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0615d06-d651-4df7-a34e-11bc0f13692e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:35.217877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c9a56-18bd-4236-b0d8-e3af8723e3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:35.219503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c9a56-18bd-4236-b0d8-e3af8723e3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:35.219940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c9a56-18bd-4236-b0d8-e3af8723e3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:35.220391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c9a56-18bd-4236-b0d8-e3af8723e3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:35.222422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c9a56-18bd-4236-b0d8-e3af8723e3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:35.224809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c9a56-18bd-4236-b0d8-e3af8723e3c6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:35.239972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 929/1867] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:35.242246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016e803-ec32-4bab-9219-63bed9e32910 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:35.242698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016e803-ec32-4bab-9219-63bed9e32910 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:35.243354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016e803-ec32-4bab-9219-63bed9e32910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:35.271059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e953ba9-6d47-4aac-abbb-981f739b57aa admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:35.276460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:35.277436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:35.277715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:35.277988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] PATH_INFO: `/v3/projects/6c39ae1e47174687bfc8e8a1000cfe51` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:35.279369 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c39ae1e47174687bfc8e8a1000cfe51)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:35.282802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:35.284692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3016e803-ec32-4bab-9219-63bed9e32910 None None] MFA Rules not processed for user `0b5d219b5998469d93dbe6d8394efb91`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:35.286573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3016e803-ec32-4bab-9219-63bed9e32910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:35.360901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:35.388015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c39ae1e47174687bfc8e8a1000cfe51', 'request_id': 'req-55b61824-1f42-461f-a8a7-98a1798cdef0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.401484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c39ae1e47174687bfc8e8a1000cfe51', 'request_id': 'req-55b61824-1f42-461f-a8a7-98a1798cdef0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.416093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c39ae1e47174687bfc8e8a1000cfe51', 'request_id': 'req-55b61824-1f42-461f-a8a7-98a1798cdef0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.440666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] The token cache is being invalidate because project 6c39ae1e47174687bfc8e8a1000cfe51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:35.441234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.442112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.448333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 939/1868] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:35 2022] DELETE /identity/v3/projects/6c39ae1e47174687bfc8e8a1000cfe51 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:35.449130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:35.451311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:35.456479 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 930/1869] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:35 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:35.457570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016e803-ec32-4bab-9219-63bed9e32910 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:35.463156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3016e803-ec32-4bab-9219-63bed9e32910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:35.490437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016e803-ec32-4bab-9219-63bed9e32910 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:35.551025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b61824-1f42-461f-a8a7-98a1798cdef0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:35.554903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:35.556568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:35.557556 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:35.557999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:35.558305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] PATH_INFO: `/v3/users/89e4c7034cfb46f0a8ba0fd4e6135e17` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:35.559500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] RBAC: Authorizing `identity:delete_user(user_id=89e4c7034cfb46f0a8ba0fd4e6135e17)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:35.561763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:35.579077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b1c87-bcca-4874-8a06-040d6d68e6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:35.580049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b1c87-bcca-4874-8a06-040d6d68e6ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:35.580474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b1c87-bcca-4874-8a06-040d6d68e6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:35.580859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b1c87-bcca-4874-8a06-040d6d68e6ed admin admin] PATH_INFO: `/v3/projects/51c276db6e4d4a2da6c201a02b09d800/users/c8aaa561314447c7a1ef0a159cc015db/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:35.601631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b1c87-bcca-4874-8a06-040d6d68e6ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c276db6e4d4a2da6c201a02b09d800, user_id=c8aaa561314447c7a1ef0a159cc015db, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:35.605688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b1c87-bcca-4874-8a06-040d6d68e6ed admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:35.623104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 931/1870] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:35 2022] PUT /identity/v3/projects/51c276db6e4d4a2da6c201a02b09d800/users/c8aaa561314447c7a1ef0a159cc015db/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:35.624879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe557-64c4-415a-8f9e-38fee4c7a292 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:35.625225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe557-64c4-415a-8f9e-38fee4c7a292 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:35.625512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe557-64c4-415a-8f9e-38fee4c7a292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:35.636441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89e4c7034cfb46f0a8ba0fd4e6135e17'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.644831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-56ffe557-64c4-415a-8f9e-38fee4c7a292 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:35.645753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89e4c7034cfb46f0a8ba0fd4e6135e17'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.646025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-56ffe557-64c4-415a-8f9e-38fee4c7a292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:35.646965 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89e4c7034cfb46f0a8ba0fd4e6135e17'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.665446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89e4c7034cfb46f0a8ba0fd4e6135e17'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.676833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89e4c7034cfb46f0a8ba0fd4e6135e17'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:35.690380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 940/1871] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:35 2022] DELETE /identity/v3/users/89e4c7034cfb46f0a8ba0fd4e6135e17 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:35.690380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:35.690380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:35.718893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 932/1872] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:35.720603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e6f23-7526-4835-abc3-8fac936b725f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:35.721087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e6f23-7526-4835-abc3-8fac936b725f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:35.721374 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e6f23-7526-4835-abc3-8fac936b725f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:35.765101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0d1e6f23-7526-4835-abc3-8fac936b725f None None] MFA Rules not processed for user `c8aaa561314447c7a1ef0a159cc015db`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:35.767532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0d1e6f23-7526-4835-abc3-8fac936b725f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:35.777972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:35.786164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766f7db-ed1e-4558-b14b-79734b60d93f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:35.796913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2637c-f4aa-409a-9bde-b16adc20f7f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:35.797902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2637c-f4aa-409a-9bde-b16adc20f7f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:35.798316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2637c-f4aa-409a-9bde-b16adc20f7f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:35.798758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2637c-f4aa-409a-9bde-b16adc20f7f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:35.801659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-49f2637c-f4aa-409a-9bde-b16adc20f7f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:35.909668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 933/1873] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:35.914745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5a98-9f84-4c0e-975f-d94917c88f11 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:35.914745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5a98-9f84-4c0e-975f-d94917c88f11 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:35.914745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5a98-9f84-4c0e-975f-d94917c88f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:35.921644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2637c-f4aa-409a-9bde-b16adc20f7f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:35.923357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2637c-f4aa-409a-9bde-b16adc20f7f9 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:35.937982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 941/1874] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:35 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:35.939284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2637c-f4aa-409a-9bde-b16adc20f7f9 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:35.941187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-49f2637c-f4aa-409a-9bde-b16adc20f7f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:35.947406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-40dd5a98-9f84-4c0e-975f-d94917c88f11 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:35.949430 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-40dd5a98-9f84-4c0e-975f-d94917c88f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:36.059141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2637c-f4aa-409a-9bde-b16adc20f7f9 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:36.069739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303eff0-67d1-45db-82fe-c87b04abeca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:36.070489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303eff0-67d1-45db-82fe-c87b04abeca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:36.071021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303eff0-67d1-45db-82fe-c87b04abeca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:36.071456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303eff0-67d1-45db-82fe-c87b04abeca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:36.071899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303eff0-67d1-45db-82fe-c87b04abeca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:36.074796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303eff0-67d1-45db-82fe-c87b04abeca2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:36.086810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:36.109493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 934/1875] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:36.110723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5a98-9f84-4c0e-975f-d94917c88f11 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:36.115791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 942/1876] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:35 2022] POST /identity/v3/projects => generated 363 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:36.116732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303eff0-67d1-45db-82fe-c87b04abeca2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:36.128002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5a98-9f84-4c0e-975f-d94917c88f11 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:36.236051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502ace7-ce48-4abd-9d1d-e54532afa97f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:36.241404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502ace7-ce48-4abd-9d1d-e54532afa97f service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:36.245261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502ace7-ce48-4abd-9d1d-e54532afa97f service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:36.246662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303eff0-67d1-45db-82fe-c87b04abeca2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:36.247698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502ace7-ce48-4abd-9d1d-e54532afa97f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:36.251532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d502ace7-ce48-4abd-9d1d-e54532afa97f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:36.256929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95970ca-eba3-4d0b-965d-8e0680950c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:36.259786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95970ca-eba3-4d0b-965d-8e0680950c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:36.260102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95970ca-eba3-4d0b-965d-8e0680950c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:36.260617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95970ca-eba3-4d0b-965d-8e0680950c87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:36.270370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95970ca-eba3-4d0b-965d-8e0680950c87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:36.270694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95970ca-eba3-4d0b-965d-8e0680950c87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:36.276775 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95970ca-eba3-4d0b-965d-8e0680950c87 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:36.304714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 943/1877] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:36.309594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95970ca-eba3-4d0b-965d-8e0680950c87 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:36.323584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95970ca-eba3-4d0b-965d-8e0680950c87 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:36.328871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e08a-ae38-4c0c-9f72-2b08e6541ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:36.330261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e08a-ae38-4c0c-9f72-2b08e6541ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:36.330563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e08a-ae38-4c0c-9f72-2b08e6541ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:36.330846 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e08a-ae38-4c0c-9f72-2b08e6541ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:36.331554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289e08a-ae38-4c0c-9f72-2b08e6541ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:36.335014 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289e08a-ae38-4c0c-9f72-2b08e6541ae6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:36.358990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502ace7-ce48-4abd-9d1d-e54532afa97f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:36.361823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502ace7-ce48-4abd-9d1d-e54532afa97f service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:36.362377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:36.375345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 944/1878] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:36 2022] POST /identity/v3/users => generated 288 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:36.376567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e08a-ae38-4c0c-9f72-2b08e6541ae6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:36.379647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 935/1879] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:36.380676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502ace7-ce48-4abd-9d1d-e54532afa97f service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:36.398487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502ace7-ce48-4abd-9d1d-e54532afa97f service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:36.403601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e08a-ae38-4c0c-9f72-2b08e6541ae6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:36.414943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e3adf-5665-4249-8c38-4342dc3fb95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:36.415705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3adf-5665-4249-8c38-4342dc3fb95e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:36.416074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3adf-5665-4249-8c38-4342dc3fb95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:36.416426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3adf-5665-4249-8c38-4342dc3fb95e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:36.418451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e3adf-5665-4249-8c38-4342dc3fb95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:36.420279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd8f05-c1a3-46c8-a29c-4e444a04ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:36.420798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e3adf-5665-4249-8c38-4342dc3fb95e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:36.421211 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd8f05-c1a3-46c8-a29c-4e444a04ce0f admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:36.421621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd8f05-c1a3-46c8-a29c-4e444a04ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:36.422008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd8f05-c1a3-46c8-a29c-4e444a04ce0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:36.423403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd8f05-c1a3-46c8-a29c-4e444a04ce0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:36.426129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd8f05-c1a3-46c8-a29c-4e444a04ce0f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:36.437892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 936/1880] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:36.438799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e3adf-5665-4249-8c38-4342dc3fb95e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:36.443119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-806e3adf-5665-4249-8c38-4342dc3fb95e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:36.480315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 945/1881] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:36 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:36.481236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd8f05-c1a3-46c8-a29c-4e444a04ce0f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:36.499204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e3adf-5665-4249-8c38-4342dc3fb95e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:36.592426 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd8f05-c1a3-46c8-a29c-4e444a04ce0f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:36.618435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22208e6-4ade-4e1f-bc28-8b21fe62110e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:36.621825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22208e6-4ade-4e1f-bc28-8b21fe62110e admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:36.627434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22208e6-4ade-4e1f-bc28-8b21fe62110e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:36.627813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22208e6-4ade-4e1f-bc28-8b21fe62110e admin admin] PATH_INFO: `/v3/projects/45c780ff6f8348ddb4dd62fa488202ad/users/a855023c40af4e31941ee1c24531f825/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:36.655537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22208e6-4ade-4e1f-bc28-8b21fe62110e admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c780ff6f8348ddb4dd62fa488202ad, user_id=a855023c40af4e31941ee1c24531f825, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:36.657129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:36.658033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22208e6-4ade-4e1f-bc28-8b21fe62110e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:36.659299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:36.659882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:36.660330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] PATH_INFO: `/v3/projects/22100922edd44f999f642bded1664109` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:36.664125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=22100922edd44f999f642bded1664109)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:36.666074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:36.675828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 946/1882] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:36 2022] PUT /identity/v3/projects/45c780ff6f8348ddb4dd62fa488202ad/users/a855023c40af4e31941ee1c24531f825/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:36.677006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22208e6-4ade-4e1f-bc28-8b21fe62110e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:36.745444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:36.769759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22100922edd44f999f642bded1664109', 'request_id': 'req-98ebcc47-3021-4c6d-9150-a53c4e505d7d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:36.780397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22100922edd44f999f642bded1664109', 'request_id': 'req-98ebcc47-3021-4c6d-9150-a53c4e505d7d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:36.795436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22100922edd44f999f642bded1664109', 'request_id': 'req-98ebcc47-3021-4c6d-9150-a53c4e505d7d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:36.814295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] The token cache is being invalidate because project 22100922edd44f999f642bded1664109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:36.814733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:36.815913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:36.819351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 937/1883] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:36 2022] DELETE /identity/v3/projects/22100922edd44f999f642bded1664109 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:36.820234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:36.824219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:36.831969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22208e6-4ade-4e1f-bc28-8b21fe62110e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:36.958364 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500c1cc-9850-45b6-be50-1dc1d4b03f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:36.959182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500c1cc-9850-45b6-be50-1dc1d4b03f67 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:36.959569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500c1cc-9850-45b6-be50-1dc1d4b03f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:36.959951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500c1cc-9850-45b6-be50-1dc1d4b03f67 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:36.960950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500c1cc-9850-45b6-be50-1dc1d4b03f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:36.963200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500c1cc-9850-45b6-be50-1dc1d4b03f67 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:36.974241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:36.983936 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebcc47-3021-4c6d-9150-a53c4e505d7d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:36.998044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 947/1884] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:36 2022] POST /identity/v3/users => generated 291 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:36.998044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500c1cc-9850-45b6-be50-1dc1d4b03f67 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:36.998044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5500c1cc-9850-45b6-be50-1dc1d4b03f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:37.002891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a767e-9f96-4b64-8907-0cd21193062d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.003949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a767e-9f96-4b64-8907-0cd21193062d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.004266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a767e-9f96-4b64-8907-0cd21193062d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.004779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a767e-9f96-4b64-8907-0cd21193062d admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.005598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a767e-9f96-4b64-8907-0cd21193062d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.008900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a767e-9f96-4b64-8907-0cd21193062d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.019527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500c1cc-9850-45b6-be50-1dc1d4b03f67 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:37.024803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.026338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 938/1885] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:37.026455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.026841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.027122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a767e-9f96-4b64-8907-0cd21193062d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:37.027308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] PATH_INFO: `/v3/users/021d7fdc72b849668ea3765c5767c109` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.028727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=021d7fdc72b849668ea3765c5767c109)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.028850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d64a767e-9f96-4b64-8907-0cd21193062d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:37.031036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.047005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a767e-9f96-4b64-8907-0cd21193062d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:37.051926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581309-0849-478f-8cea-1a906211d672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.052701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581309-0849-478f-8cea-1a906211d672 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.053056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581309-0849-478f-8cea-1a906211d672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.053407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581309-0849-478f-8cea-1a906211d672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.054204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4581309-0849-478f-8cea-1a906211d672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.055866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4581309-0849-478f-8cea-1a906211d672 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.067312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 939/1886] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:37.068200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581309-0849-478f-8cea-1a906211d672 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:37.078508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581309-0849-478f-8cea-1a906211d672 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:37.083267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdbd5e-d901-47b6-9b78-c31fa2a95d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.084191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021d7fdc72b849668ea3765c5767c109'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:37.084346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdbd5e-d901-47b6-9b78-c31fa2a95d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.084635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdbd5e-d901-47b6-9b78-c31fa2a95d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.084920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdbd5e-d901-47b6-9b78-c31fa2a95d1d admin admin] PATH_INFO: `/v3/projects/45c780ff6f8348ddb4dd62fa488202ad/users/a855023c40af4e31941ee1c24531f825/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.087378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdbd5e-d901-47b6-9b78-c31fa2a95d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c780ff6f8348ddb4dd62fa488202ad, user_id=a855023c40af4e31941ee1c24531f825, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.090275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdbd5e-d901-47b6-9b78-c31fa2a95d1d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.092375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021d7fdc72b849668ea3765c5767c109'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:37.093125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021d7fdc72b849668ea3765c5767c109'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:37.101822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 940/1887] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:37 2022] PUT /identity/v3/projects/45c780ff6f8348ddb4dd62fa488202ad/users/a855023c40af4e31941ee1c24531f825/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:37.102732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdbd5e-d901-47b6-9b78-c31fa2a95d1d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:37.115361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021d7fdc72b849668ea3765c5767c109'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:37.123897 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-42d8bf77-7625-4a77-aa91-241c780887d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021d7fdc72b849668ea3765c5767c109'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:37.130036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 948/1888] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:36 2022] DELETE /identity/v3/users/021d7fdc72b849668ea3765c5767c109 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:37.132480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f533a-4a50-46e5-8546-09a39b5fc792 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.132480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f533a-4a50-46e5-8546-09a39b5fc792 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.132480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f533a-4a50-46e5-8546-09a39b5fc792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.161430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4a2f533a-4a50-46e5-8546-09a39b5fc792 None None] MFA Rules not processed for user `a855023c40af4e31941ee1c24531f825`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:37.162657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4a2f533a-4a50-46e5-8546-09a39b5fc792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:37.175093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdbd5e-d901-47b6-9b78-c31fa2a95d1d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:37.235481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:37.247592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 949/1889] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:37 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:37.261428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f533a-4a50-46e5-8546-09a39b5fc792 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:37.263479 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4a2f533a-4a50-46e5-8546-09a39b5fc792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:37.265930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a5b67-6957-4aa6-b828-bd9f41ade3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.266744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a5b67-6957-4aa6-b828-bd9f41ade3db admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.267188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a5b67-6957-4aa6-b828-bd9f41ade3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.267568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a5b67-6957-4aa6-b828-bd9f41ade3db admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.268977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a5b67-6957-4aa6-b828-bd9f41ade3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.272239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a5b67-6957-4aa6-b828-bd9f41ade3db admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.301705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 941/1890] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:37.305465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a5b67-6957-4aa6-b828-bd9f41ade3db admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:37.307459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e84a5b67-6957-4aa6-b828-bd9f41ade3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:37.327240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a5b67-6957-4aa6-b828-bd9f41ade3db admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:37.332506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637b634-8236-4d9e-b6cc-e52dfa7eed1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.334304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637b634-8236-4d9e-b6cc-e52dfa7eed1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.334988 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637b634-8236-4d9e-b6cc-e52dfa7eed1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.335278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637b634-8236-4d9e-b6cc-e52dfa7eed1b admin admin] PATH_INFO: `/v3/projects/5c070ccc8a494b39bd4f82b01d9da8c9/users/3ca72565924f435fa0557acac43182da/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.337897 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f533a-4a50-46e5-8546-09a39b5fc792 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:37.345597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec516cc2-519f-4eac-bf2c-1927bdf4247b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.346434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec516cc2-519f-4eac-bf2c-1927bdf4247b service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.346847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec516cc2-519f-4eac-bf2c-1927bdf4247b service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.347219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec516cc2-519f-4eac-bf2c-1927bdf4247b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.357758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:37.361551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637b634-8236-4d9e-b6cc-e52dfa7eed1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c070ccc8a494b39bd4f82b01d9da8c9, user_id=3ca72565924f435fa0557acac43182da, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.361721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec516cc2-519f-4eac-bf2c-1927bdf4247b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.363499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec516cc2-519f-4eac-bf2c-1927bdf4247b service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.365478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637b634-8236-4d9e-b6cc-e52dfa7eed1b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.388596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 942/1891] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:37 2022] PUT /identity/v3/projects/5c070ccc8a494b39bd4f82b01d9da8c9/users/3ca72565924f435fa0557acac43182da/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:37.393983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 950/1892] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:37 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:34:37.402134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1acd61-16d4-419d-97ab-c4ddb92e0cce None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.402727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1acd61-16d4-419d-97ab-c4ddb92e0cce None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.403209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1acd61-16d4-419d-97ab-c4ddb92e0cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.469459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cd1acd61-16d4-419d-97ab-c4ddb92e0cce None None] MFA Rules not processed for user `3ca72565924f435fa0557acac43182da`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:37.471592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cd1acd61-16d4-419d-97ab-c4ddb92e0cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:37.488524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec516cc2-519f-4eac-bf2c-1927bdf4247b service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:37.493643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ec516cc2-519f-4eac-bf2c-1927bdf4247b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:37.510945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec516cc2-519f-4eac-bf2c-1927bdf4247b service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:37.581435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 943/1893] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:37.582168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.582706 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1acd61-16d4-419d-97ab-c4ddb92e0cce None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:37.583053 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.583346 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.583826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] PATH_INFO: `/v3/projects/9fd6cd4d3ca7423ca5490797475264f4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.585595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fd6cd4d3ca7423ca5490797475264f4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.589030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.655998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fd6cd4d3ca7423ca5490797475264f4', 'request_id': 'req-154395a9-91b0-4c3a-9e35-295b7eb4d35f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:37.665699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fd6cd4d3ca7423ca5490797475264f4', 'request_id': 'req-154395a9-91b0-4c3a-9e35-295b7eb4d35f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:37.674314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1acd61-16d4-419d-97ab-c4ddb92e0cce None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:37.678103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fd6cd4d3ca7423ca5490797475264f4', 'request_id': 'req-154395a9-91b0-4c3a-9e35-295b7eb4d35f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:37.724752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] The token cache is being invalidate because project 9fd6cd4d3ca7423ca5490797475264f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:37.726545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:37.727623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:37.739640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 951/1894] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:37 2022] DELETE /identity/v3/projects/9fd6cd4d3ca7423ca5490797475264f4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:37.741360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:37.742881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:37.755220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3f262-8e66-4797-84a2-e815835c2fa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.755992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f262-8e66-4797-84a2-e815835c2fa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.756386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f262-8e66-4797-84a2-e815835c2fa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.756785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f262-8e66-4797-84a2-e815835c2fa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.759541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cde3f262-8e66-4797-84a2-e815835c2fa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:37.794798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:37.823990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154395a9-91b0-4c3a-9e35-295b7eb4d35f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:37.829866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87582dda-0b12-4b3c-b82d-f9827c663c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.830060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87582dda-0b12-4b3c-b82d-f9827c663c41 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.830343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87582dda-0b12-4b3c-b82d-f9827c663c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.830620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87582dda-0b12-4b3c-b82d-f9827c663c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.831318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87582dda-0b12-4b3c-b82d-f9827c663c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.833344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87582dda-0b12-4b3c-b82d-f9827c663c41 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.841365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3f262-8e66-4797-84a2-e815835c2fa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.842896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3f262-8e66-4797-84a2-e815835c2fa3 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.856089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 944/1895] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:37.857203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b81015-f561-40fa-8275-d9cb89e3f3b7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.857889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b81015-f561-40fa-8275-d9cb89e3f3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.857889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b81015-f561-40fa-8275-d9cb89e3f3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.865220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 952/1896] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:37 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:37.869134 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87582dda-0b12-4b3c-b82d-f9827c663c41 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:37.877631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e9b81015-f561-40fa-8275-d9cb89e3f3b7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:37.879100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e9b81015-f561-40fa-8275-d9cb89e3f3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:37.886984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:37.953496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87582dda-0b12-4b3c-b82d-f9827c663c41 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:37.960143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dfae5-9206-4892-9295-c4a6abb639eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:37.961110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dfae5-9206-4892-9295-c4a6abb639eb admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:37.961393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dfae5-9206-4892-9295-c4a6abb639eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:37.961708 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dfae5-9206-4892-9295-c4a6abb639eb admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:37.962485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dfae5-9206-4892-9295-c4a6abb639eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:37.966486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dfae5-9206-4892-9295-c4a6abb639eb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:37.997664 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 945/1897] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:37.999347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc9d87-0927-4352-9227-520c2feab3ea None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.001123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc9d87-0927-4352-9227-520c2feab3ea None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.001450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc9d87-0927-4352-9227-520c2feab3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.009218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 953/1898] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:37 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:38.010426 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dfae5-9206-4892-9295-c4a6abb639eb admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.039623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6ecc9d87-0927-4352-9227-520c2feab3ea None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:38.044006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6ecc9d87-0927-4352-9227-520c2feab3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:38.050974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dfae5-9206-4892-9295-c4a6abb639eb admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.064471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 946/1899] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:38.065704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc9d87-0927-4352-9227-520c2feab3ea None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.074235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242fcea-25c3-4c5b-80e8-011b30c688f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.075009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242fcea-25c3-4c5b-80e8-011b30c688f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.075350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242fcea-25c3-4c5b-80e8-011b30c688f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.075559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242fcea-25c3-4c5b-80e8-011b30c688f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.076324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242fcea-25c3-4c5b-80e8-011b30c688f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.078051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242fcea-25c3-4c5b-80e8-011b30c688f1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.089744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc9d87-0927-4352-9227-520c2feab3ea None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.095852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 954/1900] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:38.097556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbf1e0-9f14-4e7b-8cce-0d4fe3dafea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.101361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242fcea-25c3-4c5b-80e8-011b30c688f1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.101922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbf1e0-9f14-4e7b-8cce-0d4fe3dafea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.102646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbf1e0-9f14-4e7b-8cce-0d4fe3dafea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.102989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbf1e0-9f14-4e7b-8cce-0d4fe3dafea8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.104493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbf1e0-9f14-4e7b-8cce-0d4fe3dafea8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:38.105741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbf1e0-9f14-4e7b-8cce-0d4fe3dafea8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.107445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbf1e0-9f14-4e7b-8cce-0d4fe3dafea8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.118123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242fcea-25c3-4c5b-80e8-011b30c688f1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.125438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491a9db-e258-4d7a-946b-4dc5d23946c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.127240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491a9db-e258-4d7a-946b-4dc5d23946c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.127715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491a9db-e258-4d7a-946b-4dc5d23946c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.128133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491a9db-e258-4d7a-946b-4dc5d23946c4 admin admin] PATH_INFO: `/v3/projects/f02f5ba2d0f34464bef6f6d30830766f/users/4f8df9c072ce45b58a1f49330a73c292/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.128602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 947/1901] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:38.133772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbf1e0-9f14-4e7b-8cce-0d4fe3dafea8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.147544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbf1e0-9f14-4e7b-8cce-0d4fe3dafea8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.155156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd57cce-9b8e-4cd3-84ca-0232474e5b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.155940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd57cce-9b8e-4cd3-84ca-0232474e5b34 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.156322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd57cce-9b8e-4cd3-84ca-0232474e5b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.157075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd57cce-9b8e-4cd3-84ca-0232474e5b34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.157868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd57cce-9b8e-4cd3-84ca-0232474e5b34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.160497 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491a9db-e258-4d7a-946b-4dc5d23946c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02f5ba2d0f34464bef6f6d30830766f, user_id=4f8df9c072ce45b58a1f49330a73c292, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.160641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd57cce-9b8e-4cd3-84ca-0232474e5b34 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.165961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491a9db-e258-4d7a-946b-4dc5d23946c4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.197457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 955/1902] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:38 2022] PUT /identity/v3/projects/f02f5ba2d0f34464bef6f6d30830766f/users/4f8df9c072ce45b58a1f49330a73c292/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:38.202383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491a9db-e258-4d7a-946b-4dc5d23946c4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.208774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 948/1903] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:38 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:38.209831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd57cce-9b8e-4cd3-84ca-0232474e5b34 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.288584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491a9db-e258-4d7a-946b-4dc5d23946c4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.305950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd57cce-9b8e-4cd3-84ca-0232474e5b34 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.311482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea3854-6e9d-4b79-98ec-f9af79078f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.312173 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea3854-6e9d-4b79-98ec-f9af79078f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.312501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea3854-6e9d-4b79-98ec-f9af79078f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.312779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea3854-6e9d-4b79-98ec-f9af79078f9e admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.313473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea3854-6e9d-4b79-98ec-f9af79078f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.315556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea3854-6e9d-4b79-98ec-f9af79078f9e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.341513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 949/1904] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:38 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:38.346794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea3854-6e9d-4b79-98ec-f9af79078f9e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.358574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea3854-6e9d-4b79-98ec-f9af79078f9e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.366218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c1677-f629-41c7-bd4b-490839671b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.367350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c1677-f629-41c7-bd4b-490839671b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.367350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c1677-f629-41c7-bd4b-490839671b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.367350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c1677-f629-41c7-bd4b-490839671b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.368288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c1677-f629-41c7-bd4b-490839671b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.370132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c1677-f629-41c7-bd4b-490839671b8d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.373202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:38.384771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 950/1905] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:34:38.389103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c1677-f629-41c7-bd4b-490839671b8d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.400428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcdd20-c95a-4793-a0f6-7002107d3e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.402281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcdd20-c95a-4793-a0f6-7002107d3e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.402770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcdd20-c95a-4793-a0f6-7002107d3e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.403230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcdd20-c95a-4793-a0f6-7002107d3e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.404175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcdd20-c95a-4793-a0f6-7002107d3e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.406841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c1677-f629-41c7-bd4b-490839671b8d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.408195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcdd20-c95a-4793-a0f6-7002107d3e10 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.414879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f364522-91c7-467d-82ce-4ec94ce39f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.416215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f364522-91c7-467d-82ce-4ec94ce39f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.417556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f364522-91c7-467d-82ce-4ec94ce39f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.417884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f364522-91c7-467d-82ce-4ec94ce39f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.420456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f364522-91c7-467d-82ce-4ec94ce39f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.422934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 956/1906] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:38.425411 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f364522-91c7-467d-82ce-4ec94ce39f30 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.428202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcdd20-c95a-4793-a0f6-7002107d3e10 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.446674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 951/1907] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:34:38.451570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcdd20-c95a-4793-a0f6-7002107d3e10 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.453395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f364522-91c7-467d-82ce-4ec94ce39f30 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.462963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d43e6-a6d8-4c99-9469-49769a96a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.464684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d43e6-a6d8-4c99-9469-49769a96a651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.465326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d43e6-a6d8-4c99-9469-49769a96a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.465612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d43e6-a6d8-4c99-9469-49769a96a651 admin admin] PATH_INFO: `/v3/projects/f02f5ba2d0f34464bef6f6d30830766f/users/4f8df9c072ce45b58a1f49330a73c292/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.471376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d43e6-a6d8-4c99-9469-49769a96a651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02f5ba2d0f34464bef6f6d30830766f, user_id=4f8df9c072ce45b58a1f49330a73c292, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.474371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d43e6-a6d8-4c99-9469-49769a96a651 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.479771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f364522-91c7-467d-82ce-4ec94ce39f30 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.483604 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:38.490283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf9004-7e63-41bb-8c58-6b067e471d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.491519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf9004-7e63-41bb-8c58-6b067e471d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.494032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf9004-7e63-41bb-8c58-6b067e471d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.494347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf9004-7e63-41bb-8c58-6b067e471d53 admin admin] PATH_INFO: `/v3/projects/ce5eaf395f2045a988c7a4d523b024d6/users/555c49020bb34da7be0d00592c3159a6/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.505072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 957/1908] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:38 2022] PUT /identity/v3/projects/f02f5ba2d0f34464bef6f6d30830766f/users/4f8df9c072ce45b58a1f49330a73c292/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:38.512804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5db00-c54c-4483-8961-cf477d6a3fb5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.513094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5db00-c54c-4483-8961-cf477d6a3fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.513383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5db00-c54c-4483-8961-cf477d6a3fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.524870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf9004-7e63-41bb-8c58-6b067e471d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce5eaf395f2045a988c7a4d523b024d6, user_id=555c49020bb34da7be0d00592c3159a6, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.530080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf9004-7e63-41bb-8c58-6b067e471d53 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.550314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 952/1909] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:38 2022] PUT /identity/v3/projects/ce5eaf395f2045a988c7a4d523b024d6/users/555c49020bb34da7be0d00592c3159a6/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:38.562858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafeb82c-5d54-4c87-b245-6a6565ff3306 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.563185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafeb82c-5d54-4c87-b245-6a6565ff3306 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.563868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafeb82c-5d54-4c87-b245-6a6565ff3306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.580320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1da5db00-c54c-4483-8961-cf477d6a3fb5 None None] MFA Rules not processed for user `4f8df9c072ce45b58a1f49330a73c292`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:38.581857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1da5db00-c54c-4483-8961-cf477d6a3fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:38.604710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fafeb82c-5d54-4c87-b245-6a6565ff3306 None None] MFA Rules not processed for user `555c49020bb34da7be0d00592c3159a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:38.606127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fafeb82c-5d54-4c87-b245-6a6565ff3306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:38.682242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 958/1910] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:38.695231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5db00-c54c-4483-8961-cf477d6a3fb5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.697036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1da5db00-c54c-4483-8961-cf477d6a3fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:38.702280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 953/1911] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:38.706963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafeb82c-5d54-4c87-b245-6a6565ff3306 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.783840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5db00-c54c-4483-8961-cf477d6a3fb5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.794890 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafeb82c-5d54-4c87-b245-6a6565ff3306 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:38.799660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dfe3c-9065-41cd-aca5-6adfac5b5e47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.803885 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dfe3c-9065-41cd-aca5-6adfac5b5e47 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.804191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dfe3c-9065-41cd-aca5-6adfac5b5e47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.804769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dfe3c-9065-41cd-aca5-6adfac5b5e47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.815668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d17b3-a514-44fa-8e28-9711e11575cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:38.816405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d17b3-a514-44fa-8e28-9711e11575cd admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:38.816684 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d17b3-a514-44fa-8e28-9711e11575cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:38.816962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d17b3-a514-44fa-8e28-9711e11575cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:38.817660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d17b3-a514-44fa-8e28-9711e11575cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.819569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d17b3-a514-44fa-8e28-9711e11575cd admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.837268 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dfe3c-9065-41cd-aca5-6adfac5b5e47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:38.839418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dfe3c-9065-41cd-aca5-6adfac5b5e47 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:38.868218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 959/1912] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:38.869132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dfe3c-9065-41cd-aca5-6adfac5b5e47 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:38.869433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 954/1913] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:38 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:38.881115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dfe3c-9065-41cd-aca5-6adfac5b5e47 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:39.002709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d17b3-a514-44fa-8e28-9711e11575cd admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:39.017474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8008ced-91f8-4638-8666-e67077de0f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:39.018155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008ced-91f8-4638-8666-e67077de0f80 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:39.018443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008ced-91f8-4638-8666-e67077de0f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:39.018734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008ced-91f8-4638-8666-e67077de0f80 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:39.019400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8008ced-91f8-4638-8666-e67077de0f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:39.031421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8008ced-91f8-4638-8666-e67077de0f80 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:39.074109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:39.085420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 960/1914] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:38 2022] POST /identity/v3/users => generated 298 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:39.091486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8008ced-91f8-4638-8666-e67077de0f80 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:39.107913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d17b3-a514-44fa-8e28-9711e11575cd admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:39.115437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8008ced-91f8-4638-8666-e67077de0f80 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:39.118830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78fb80-4d4d-4dbb-821c-8107b355cb97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:39.119765 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78fb80-4d4d-4dbb-821c-8107b355cb97 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:39.120057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78fb80-4d4d-4dbb-821c-8107b355cb97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:39.120367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78fb80-4d4d-4dbb-821c-8107b355cb97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:39.125447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3b78fb80-4d4d-4dbb-821c-8107b355cb97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:39.131030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8838a5-23ea-43fd-b28f-7884df1de5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:39.131985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8838a5-23ea-43fd-b28f-7884df1de5ed admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:39.132274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8838a5-23ea-43fd-b28f-7884df1de5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:39.132635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8838a5-23ea-43fd-b28f-7884df1de5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:39.134121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8838a5-23ea-43fd-b28f-7884df1de5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:39.138833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8838a5-23ea-43fd-b28f-7884df1de5ed admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:39.159200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 961/1915] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:34:39.160571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8838a5-23ea-43fd-b28f-7884df1de5ed admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:39.166033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:39.175459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8838a5-23ea-43fd-b28f-7884df1de5ed admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:39.180917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bfe54-6f19-44b5-bab7-2fd5d78ef320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:39.182734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bfe54-6f19-44b5-bab7-2fd5d78ef320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:39.182734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bfe54-6f19-44b5-bab7-2fd5d78ef320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:39.182930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bfe54-6f19-44b5-bab7-2fd5d78ef320 admin admin] PATH_INFO: `/v3/projects/9aeb05965264452a82593d4bbe229cd9/users/85365f236cb140859df14794e92139fe/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:39.204984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bfe54-6f19-44b5-bab7-2fd5d78ef320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aeb05965264452a82593d4bbe229cd9, user_id=85365f236cb140859df14794e92139fe, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:39.207917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bfe54-6f19-44b5-bab7-2fd5d78ef320 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:39.228647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 962/1916] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:39 2022] PUT /identity/v3/projects/9aeb05965264452a82593d4bbe229cd9/users/85365f236cb140859df14794e92139fe/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:39.232739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bfe54-6f19-44b5-bab7-2fd5d78ef320 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:39.261181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78fb80-4d4d-4dbb-821c-8107b355cb97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:39.262634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78fb80-4d4d-4dbb-821c-8107b355cb97 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:39.284589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 955/1917] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:39.360048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bfe54-6f19-44b5-bab7-2fd5d78ef320 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:39.375330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5ca2d-5750-4859-8b50-09f995373149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:39.378021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5ca2d-5750-4859-8b50-09f995373149 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:39.378315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5ca2d-5750-4859-8b50-09f995373149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:39.378599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5ca2d-5750-4859-8b50-09f995373149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:39.383317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5ca2d-5750-4859-8b50-09f995373149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:39.390665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5ca2d-5750-4859-8b50-09f995373149 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:39.417610 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 963/1918] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:39.421230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78fb80-4d4d-4dbb-821c-8107b355cb97 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:39.438966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78fb80-4d4d-4dbb-821c-8107b355cb97 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:39.447351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc3957-7160-442c-a68c-750beb9c965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:39.448640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc3957-7160-442c-a68c-750beb9c965a admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:39.449033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc3957-7160-442c-a68c-750beb9c965a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:39.449186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc3957-7160-442c-a68c-750beb9c965a admin admin] PATH_INFO: `/v3/projects/9aeb05965264452a82593d4bbe229cd9/users/85365f236cb140859df14794e92139fe/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:39.461654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc3957-7160-442c-a68c-750beb9c965a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aeb05965264452a82593d4bbe229cd9, user_id=85365f236cb140859df14794e92139fe, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:39.464045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc3957-7160-442c-a68c-750beb9c965a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:39.501441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 956/1919] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:39 2022] PUT /identity/v3/projects/9aeb05965264452a82593d4bbe229cd9/users/85365f236cb140859df14794e92139fe/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:39.503697 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fb05d-cd62-47bd-bb03-94c1b0b29dc9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:39.504286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fb05d-cd62-47bd-bb03-94c1b0b29dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:39.504853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fb05d-cd62-47bd-bb03-94c1b0b29dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:39.563679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-be8fb05d-cd62-47bd-bb03-94c1b0b29dc9 None None] MFA Rules not processed for user `85365f236cb140859df14794e92139fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:39.570098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-be8fb05d-cd62-47bd-bb03-94c1b0b29dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:39.738406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 964/1920] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:39.766999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc3957-7160-442c-a68c-750beb9c965a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:39.898453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc3957-7160-442c-a68c-750beb9c965a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:39.907034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a31754-5520-44f2-b996-cc9311c0eea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:39.907720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a31754-5520-44f2-b996-cc9311c0eea2 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:39.908010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a31754-5520-44f2-b996-cc9311c0eea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:39.908289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a31754-5520-44f2-b996-cc9311c0eea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:39.923829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a31754-5520-44f2-b996-cc9311c0eea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:39.925329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a31754-5520-44f2-b996-cc9311c0eea2 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:39.955542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 957/1921] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:40.089998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fb05d-cd62-47bd-bb03-94c1b0b29dc9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:40.100409 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fb05d-cd62-47bd-bb03-94c1b0b29dc9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:40.105386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efc6e6-28f6-48f0-ba07-c0919161318c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:40.106055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc6e6-28f6-48f0-ba07-c0919161318c service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:40.106343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc6e6-28f6-48f0-ba07-c0919161318c service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:40.106655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc6e6-28f6-48f0-ba07-c0919161318c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:40.175119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efc6e6-28f6-48f0-ba07-c0919161318c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:40.176648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efc6e6-28f6-48f0-ba07-c0919161318c service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:40.180273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:40.187513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 965/1922] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:40.518859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a31754-5520-44f2-b996-cc9311c0eea2 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:40.600780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a31754-5520-44f2-b996-cc9311c0eea2 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:40.607361 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:40.609593 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:40.610179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:40.610816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] PATH_INFO: `/v3/users/555c49020bb34da7be0d00592c3159a6` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:40.613256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=555c49020bb34da7be0d00592c3159a6)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:40.615880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:40.651762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:40.697475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555c49020bb34da7be0d00592c3159a6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:40.699138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555c49020bb34da7be0d00592c3159a6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:40.700202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555c49020bb34da7be0d00592c3159a6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:40.714115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555c49020bb34da7be0d00592c3159a6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:40.733298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555c49020bb34da7be0d00592c3159a6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:40.756149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 958/1923] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:40 2022] DELETE /identity/v3/users/555c49020bb34da7be0d00592c3159a6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:40.760889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4ac84-f68b-44a0-85a0-77e3bc7dd4f6 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:40.761424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4ac84-f68b-44a0-85a0-77e3bc7dd4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:40.761798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4ac84-f68b-44a0-85a0-77e3bc7dd4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:40.782981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6ee4ac84-f68b-44a0-85a0-77e3bc7dd4f6 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:40.784637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6ee4ac84-f68b-44a0-85a0-77e3bc7dd4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:40.863657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 966/1924] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:40.877926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:40.879387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:40.941748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a234fca-df6b-4f6e-bfb2-b0f29b1bcd05 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:40.945292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb56d-7efb-4593-9a03-1b461fdf40ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:40.945937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb56d-7efb-4593-9a03-1b461fdf40ad service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:40.946217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb56d-7efb-4593-9a03-1b461fdf40ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:40.946491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb56d-7efb-4593-9a03-1b461fdf40ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:40.956573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bb56d-7efb-4593-9a03-1b461fdf40ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:40.958084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bb56d-7efb-4593-9a03-1b461fdf40ad service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:40.971967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 959/1925] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:41.415807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4ac84-f68b-44a0-85a0-77e3bc7dd4f6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:41.418002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6ee4ac84-f68b-44a0-85a0-77e3bc7dd4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:41.445920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4ac84-f68b-44a0-85a0-77e3bc7dd4f6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:41.458029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:41.458930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:41.459219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:41.460340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] PATH_INFO: `/v3/projects/ce5eaf395f2045a988c7a4d523b024d6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:41.461762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce5eaf395f2045a988c7a4d523b024d6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:41.464674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:41.576376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce5eaf395f2045a988c7a4d523b024d6', 'request_id': 'req-d17fb339-eeee-4d33-8352-796aaf67d4c8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.585099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce5eaf395f2045a988c7a4d523b024d6', 'request_id': 'req-d17fb339-eeee-4d33-8352-796aaf67d4c8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.598435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce5eaf395f2045a988c7a4d523b024d6', 'request_id': 'req-d17fb339-eeee-4d33-8352-796aaf67d4c8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.637995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] The token cache is being invalidate because project ce5eaf395f2045a988c7a4d523b024d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:41.638281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.638960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.643083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 967/1926] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:41 2022] DELETE /identity/v3/projects/ce5eaf395f2045a988c7a4d523b024d6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:41.648091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb56d-7efb-4593-9a03-1b461fdf40ad service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:41.664281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb56d-7efb-4593-9a03-1b461fdf40ad service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:41.713808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:41.716041 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:41.775904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:41.777401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:41.777401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:41.777658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] PATH_INFO: `/v3/users/85365f236cb140859df14794e92139fe` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:41.785194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=85365f236cb140859df14794e92139fe)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:41.788530 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:41.802337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fb339-eeee-4d33-8352-796aaf67d4c8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:41.809381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:41.810271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:41.810554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:41.810810 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] PATH_INFO: `/v3/users/db532044d18b4370b72b59757909b789` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:41.812654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=db532044d18b4370b72b59757909b789)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:41.816047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:41.852380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:41.893861 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:41.896086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85365f236cb140859df14794e92139fe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.897019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85365f236cb140859df14794e92139fe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.898015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85365f236cb140859df14794e92139fe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.913268 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db532044d18b4370b72b59757909b789'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.918634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85365f236cb140859df14794e92139fe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.935812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-919426c0-3e8d-40a5-8d52-d7f3ffad0d5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85365f236cb140859df14794e92139fe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.949611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db532044d18b4370b72b59757909b789'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.950783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db532044d18b4370b72b59757909b789'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.982007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db532044d18b4370b72b59757909b789'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:41.983918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 960/1927] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:41 2022] DELETE /identity/v3/users/85365f236cb140859df14794e92139fe => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:41.999023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fd803eb8-78c0-471d-bdc8-851c46d02a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db532044d18b4370b72b59757909b789'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.009395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 968/1928] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:41 2022] DELETE /identity/v3/users/db532044d18b4370b72b59757909b789 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:42.016801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbf2e2-035b-46d1-910a-012730940be7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.017358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbf2e2-035b-46d1-910a-012730940be7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.017849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbf2e2-035b-46d1-910a-012730940be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:42.039333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4a3b6-7c9f-4efd-ab4a-7a3432a7c161 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.039635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4a3b6-7c9f-4efd-ab4a-7a3432a7c161 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.039919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4a3b6-7c9f-4efd-ab4a-7a3432a7c161 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:42.042541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 969/1929] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 17:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 17:34:42.051049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2bfbf2e2-035b-46d1-910a-012730940be7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:42.052403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2bfbf2e2-035b-46d1-910a-012730940be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.060784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de0cf3-63da-4a02-b2f4-19d0127f3ef7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.061106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de0cf3-63da-4a02-b2f4-19d0127f3ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.061392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de0cf3-63da-4a02-b2f4-19d0127f3ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:42.106440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-02de0cf3-63da-4a02-b2f4-19d0127f3ef7 None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:42.108911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-02de0cf3-63da-4a02-b2f4-19d0127f3ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.208314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 961/1930] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:42.227715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbf2e2-035b-46d1-910a-012730940be7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:42.230092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2bfbf2e2-035b-46d1-910a-012730940be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.271421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 970/1931] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 17:34:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:42.330289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbf2e2-035b-46d1-910a-012730940be7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:42.348590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6c98f-653c-478b-8479-188755ce3af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:42.349891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6c98f-653c-478b-8479-188755ce3af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.357221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6c98f-653c-478b-8479-188755ce3af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.357729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6c98f-653c-478b-8479-188755ce3af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:42.366916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de0cf3-63da-4a02-b2f4-19d0127f3ef7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:42.378964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-02de0cf3-63da-4a02-b2f4-19d0127f3ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.381273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6c98f-653c-478b-8479-188755ce3af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:42.383457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6c98f-653c-478b-8479-188755ce3af1 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:42.394032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de0cf3-63da-4a02-b2f4-19d0127f3ef7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:42.399188 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 962/1932] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:34:42.401992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849802ef-ae0a-4257-bb8c-19038175e3c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:42.402741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849802ef-ae0a-4257-bb8c-19038175e3c3 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.403116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849802ef-ae0a-4257-bb8c-19038175e3c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.403460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849802ef-ae0a-4257-bb8c-19038175e3c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:42.414091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849802ef-ae0a-4257-bb8c-19038175e3c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:42.415820 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849802ef-ae0a-4257-bb8c-19038175e3c3 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:42.428607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 971/1933] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:34:42.535059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6c98f-653c-478b-8479-188755ce3af1 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:42.536990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8fc6c98f-653c-478b-8479-188755ce3af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.557726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6c98f-653c-478b-8479-188755ce3af1 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:42.563248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:42.564218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.564612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.565095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] PATH_INFO: `/v3/users/3ca72565924f435fa0557acac43182da` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:42.566427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ca72565924f435fa0557acac43182da)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:42.568471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:42.634527 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ca72565924f435fa0557acac43182da'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.637535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ca72565924f435fa0557acac43182da'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.638451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ca72565924f435fa0557acac43182da'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.651390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ca72565924f435fa0557acac43182da'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.656897 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:42.662249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849802ef-ae0a-4257-bb8c-19038175e3c3 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:42.663762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-849802ef-ae0a-4257-bb8c-19038175e3c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.671309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a7d9dfd7-12a0-4ca4-ac57-00ecc5a767cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ca72565924f435fa0557acac43182da'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.690911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849802ef-ae0a-4257-bb8c-19038175e3c3 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:42.703639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:42.703639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.703639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.703639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] PATH_INFO: `/v3/projects/9aeb05965264452a82593d4bbe229cd9` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:42.704228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aeb05965264452a82593d4bbe229cd9)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:42.704982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 963/1934] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:42 2022] DELETE /identity/v3/users/3ca72565924f435fa0557acac43182da => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:42.706823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:42.707552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba79ec3-32fa-49e7-a9fc-6e046b899fbc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.709745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba79ec3-32fa-49e7-a9fc-6e046b899fbc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.710499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba79ec3-32fa-49e7-a9fc-6e046b899fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:42.739266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-9ba79ec3-32fa-49e7-a9fc-6e046b899fbc None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:42.742161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9ba79ec3-32fa-49e7-a9fc-6e046b899fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.780340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aeb05965264452a82593d4bbe229cd9', 'request_id': 'req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.786901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aeb05965264452a82593d4bbe229cd9', 'request_id': 'req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.809924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aeb05965264452a82593d4bbe229cd9', 'request_id': 'req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.823569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] The token cache is being invalidate because project 9aeb05965264452a82593d4bbe229cd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:42.823893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.824507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4bc41938-2210-4f2e-b8bc-3eeb302ef4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:42.829576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 972/1935] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:42 2022] DELETE /identity/v3/projects/9aeb05965264452a82593d4bbe229cd9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:42.830837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c30db6-b8cc-4c8f-b47b-86d351ce9f11 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.831127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c30db6-b8cc-4c8f-b47b-86d351ce9f11 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.836889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c30db6-b8cc-4c8f-b47b-86d351ce9f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:42.837031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 964/1936] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:42.847011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba79ec3-32fa-49e7-a9fc-6e046b899fbc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:42.848398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9ba79ec3-32fa-49e7-a9fc-6e046b899fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.849004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:42.867214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c8c30db6-b8cc-4c8f-b47b-86d351ce9f11 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:42.869059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c8c30db6-b8cc-4c8f-b47b-86d351ce9f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.926767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba79ec3-32fa-49e7-a9fc-6e046b899fbc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:42.943798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 973/1937] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:42.952067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85058bb5-de57-4ac4-91a8-9b017603fd5e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.952648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85058bb5-de57-4ac4-91a8-9b017603fd5e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.952648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85058bb5-de57-4ac4-91a8-9b017603fd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:42.972732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-85058bb5-de57-4ac4-91a8-9b017603fd5e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:42.974806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-85058bb5-de57-4ac4-91a8-9b017603fd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.984951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 974/1938] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:42.985886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85058bb5-de57-4ac4-91a8-9b017603fd5e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:42.987157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-85058bb5-de57-4ac4-91a8-9b017603fd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:42.988650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ff89e-db93-4c5b-9ba2-470ebffa593a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:42.989316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ff89e-db93-4c5b-9ba2-470ebffa593a service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:42.989603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ff89e-db93-4c5b-9ba2-470ebffa593a service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:42.989882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ff89e-db93-4c5b-9ba2-470ebffa593a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.001202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85058bb5-de57-4ac4-91a8-9b017603fd5e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.001342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ff89e-db93-4c5b-9ba2-470ebffa593a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.005741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ff89e-db93-4c5b-9ba2-470ebffa593a service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.006530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.007401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.007684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.007963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] PATH_INFO: `/v3/projects/932b6c44526b445fa0977d5614687bf2` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.009206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=932b6c44526b445fa0977d5614687bf2)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.012721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.018279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 965/1939] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:43.019158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ff89e-db93-4c5b-9ba2-470ebffa593a service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.030066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ff89e-db93-4c5b-9ba2-470ebffa593a service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.035378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb283c0-458e-425b-b14c-0c3dec851982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.035785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb283c0-458e-425b-b14c-0c3dec851982 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.036071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb283c0-458e-425b-b14c-0c3dec851982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.036380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb283c0-458e-425b-b14c-0c3dec851982 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.037114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb283c0-458e-425b-b14c-0c3dec851982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:43.037430 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb283c0-458e-425b-b14c-0c3dec851982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.039015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb283c0-458e-425b-b14c-0c3dec851982 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.057503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 966/1940] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:43.059301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb283c0-458e-425b-b14c-0c3dec851982 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.071277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb283c0-458e-425b-b14c-0c3dec851982 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.075793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932b6c44526b445fa0977d5614687bf2', 'request_id': 'req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.083184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932b6c44526b445fa0977d5614687bf2', 'request_id': 'req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.099941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:43.100102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932b6c44526b445fa0977d5614687bf2', 'request_id': 'req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.117217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] The token cache is being invalidate because project 932b6c44526b445fa0977d5614687bf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:43.117511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.118429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.120044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 975/1941] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:42 2022] DELETE /identity/v3/projects/932b6c44526b445fa0977d5614687bf2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:43.124131 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.125787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:43.169919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2576d57-2f5a-4997-ae87-c01145de4709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.171150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576d57-2f5a-4997-ae87-c01145de4709 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.171434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576d57-2f5a-4997-ae87-c01145de4709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.171710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576d57-2f5a-4997-ae87-c01145de4709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.172576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2576d57-2f5a-4997-ae87-c01145de4709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.175445 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2576d57-2f5a-4997-ae87-c01145de4709 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.206277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 967/1942] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:43 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:43.210471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2576d57-2f5a-4997-ae87-c01145de4709 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.212116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e2576d57-2f5a-4997-ae87-c01145de4709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:43.215711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15020e4f-5a3f-44b3-9610-cdffa1f7ca04 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.246888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:43.288535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.289419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.289701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.289977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] PATH_INFO: `/v3/users/0b5d219b5998469d93dbe6d8394efb91` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.291816 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5d219b5998469d93dbe6d8394efb91)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.292596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2576d57-2f5a-4997-ae87-c01145de4709 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.294140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.297884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c55746-4c9c-4281-9853-cc48b4316ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.298928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c55746-4c9c-4281-9853-cc48b4316ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.299367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c55746-4c9c-4281-9853-cc48b4316ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.299652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c55746-4c9c-4281-9853-cc48b4316ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.300373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c55746-4c9c-4281-9853-cc48b4316ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.302283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c55746-4c9c-4281-9853-cc48b4316ef9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.327830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 968/1943] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:43 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:34:43.331737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c55746-4c9c-4281-9853-cc48b4316ef9 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.344710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c55746-4c9c-4281-9853-cc48b4316ef9 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.350285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf551f-d061-4773-8fe9-fbed7d58e3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.350972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf551f-d061-4773-8fe9-fbed7d58e3ba admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.351257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf551f-d061-4773-8fe9-fbed7d58e3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.352081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf551f-d061-4773-8fe9-fbed7d58e3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.353008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf551f-d061-4773-8fe9-fbed7d58e3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.356509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5d219b5998469d93dbe6d8394efb91'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.357388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf551f-d061-4773-8fe9-fbed7d58e3ba admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.373156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 969/1944] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:34:43.373991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf551f-d061-4773-8fe9-fbed7d58e3ba admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.375708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bfbf551f-d061-4773-8fe9-fbed7d58e3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:43.375814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5d219b5998469d93dbe6d8394efb91'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.376945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5d219b5998469d93dbe6d8394efb91'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.389462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf551f-d061-4773-8fe9-fbed7d58e3ba admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.394499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.395370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.395672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.395929 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] PATH_INFO: `/v3/projects/5c070ccc8a494b39bd4f82b01d9da8c9` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.397164 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c070ccc8a494b39bd4f82b01d9da8c9)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.399089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.404037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5d219b5998469d93dbe6d8394efb91'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.414924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5d219b5998469d93dbe6d8394efb91'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.425715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 976/1945] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:43 2022] DELETE /identity/v3/users/0b5d219b5998469d93dbe6d8394efb91 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:43.426707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.428148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:43.458428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c070ccc8a494b39bd4f82b01d9da8c9', 'request_id': 'req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.466384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c070ccc8a494b39bd4f82b01d9da8c9', 'request_id': 'req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.472440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c070ccc8a494b39bd4f82b01d9da8c9', 'request_id': 'req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.485380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] The token cache is being invalidate because project 5c070ccc8a494b39bd4f82b01d9da8c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:43.485675 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.486369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.490842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 970/1946] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:43 2022] DELETE /identity/v3/projects/5c070ccc8a494b39bd4f82b01d9da8c9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:43.492175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.493759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:43.508862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:43.521340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de70d9-1cdc-4ecf-ae57-55816e3ad61f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.569270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363b1d-27a0-4dd5-9eb2-ad6ce5fc3394 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.575097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.576140 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.576466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.576737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] PATH_INFO: `/v3/users/4f8df9c072ce45b58a1f49330a73c292` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.578196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8df9c072ce45b58a1f49330a73c292)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.579839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:43.580141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.591936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80c48c-3a67-42b9-9ec1-c7a833be6900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.592664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80c48c-3a67-42b9-9ec1-c7a833be6900 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.592950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80c48c-3a67-42b9-9ec1-c7a833be6900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.593384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80c48c-3a67-42b9-9ec1-c7a833be6900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.594304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80c48c-3a67-42b9-9ec1-c7a833be6900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.596908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80c48c-3a67-42b9-9ec1-c7a833be6900 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.611716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 977/1947] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:43.613369 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80c48c-3a67-42b9-9ec1-c7a833be6900 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.615474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ee80c48c-3a67-42b9-9ec1-c7a833be6900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:43.630238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80c48c-3a67-42b9-9ec1-c7a833be6900 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.635114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaace5d-984a-4b2c-b6ad-dbbeac2d7529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.636202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaace5d-984a-4b2c-b6ad-dbbeac2d7529 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.636517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaace5d-984a-4b2c-b6ad-dbbeac2d7529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.636806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaace5d-984a-4b2c-b6ad-dbbeac2d7529 admin admin] PATH_INFO: `/v3/projects/2bbd90390a0242dd8be8b1bbbabcaf2a/users/11e60ca4cf7a471c9113b801c8fd103c/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.645676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8df9c072ce45b58a1f49330a73c292'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.646565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8df9c072ce45b58a1f49330a73c292'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.647233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8df9c072ce45b58a1f49330a73c292'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.653419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaace5d-984a-4b2c-b6ad-dbbeac2d7529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bbd90390a0242dd8be8b1bbbabcaf2a, user_id=11e60ca4cf7a471c9113b801c8fd103c, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.655891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8df9c072ce45b58a1f49330a73c292'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.656361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaace5d-984a-4b2c-b6ad-dbbeac2d7529 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.673554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 978/1948] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:43 2022] PUT /identity/v3/projects/2bbd90390a0242dd8be8b1bbbabcaf2a/users/11e60ca4cf7a471c9113b801c8fd103c/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:43.676830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8df9c072ce45b58a1f49330a73c292'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:43.684392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14d84b-dd94-4ddb-b2c2-4d96dd8af165 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.684748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14d84b-dd94-4ddb-b2c2-4d96dd8af165 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.684877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14d84b-dd94-4ddb-b2c2-4d96dd8af165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.693889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 971/1949] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:43 2022] DELETE /identity/v3/users/4f8df9c072ce45b58a1f49330a73c292 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:43.721201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0a14d84b-dd94-4ddb-b2c2-4d96dd8af165 None None] MFA Rules not processed for user `11e60ca4cf7a471c9113b801c8fd103c`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:43.722971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0a14d84b-dd94-4ddb-b2c2-4d96dd8af165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:43.800539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 979/1950] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:43.813304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.814877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:43.907860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541f989-f928-4089-bfd6-145a82bfdde1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:43.913254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e66c95-5f23-47d9-9145-5facb065282a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:43.913919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e66c95-5f23-47d9-9145-5facb065282a service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:43.914205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e66c95-5f23-47d9-9145-5facb065282a service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:43.914495 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e66c95-5f23-47d9-9145-5facb065282a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:43.927126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e66c95-5f23-47d9-9145-5facb065282a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:43.928807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e66c95-5f23-47d9-9145-5facb065282a service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:43.942573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 972/1951] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:43.962793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14d84b-dd94-4ddb-b2c2-4d96dd8af165 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:43.964947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0a14d84b-dd94-4ddb-b2c2-4d96dd8af165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:44.046926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14d84b-dd94-4ddb-b2c2-4d96dd8af165 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.051624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.052608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.053065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.053347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] PATH_INFO: `/v3/users/11e60ca4cf7a471c9113b801c8fd103c` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.054806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e60ca4cf7a471c9113b801c8fd103c)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.057666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.139281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e60ca4cf7a471c9113b801c8fd103c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.153093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e60ca4cf7a471c9113b801c8fd103c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.154226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e60ca4cf7a471c9113b801c8fd103c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.155463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:44.173593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e60ca4cf7a471c9113b801c8fd103c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.188384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e60ca4cf7a471c9113b801c8fd103c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.205663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887ed95-6ecf-4506-8a70-904d6b4f1df1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.206033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887ed95-6ecf-4506-8a70-904d6b4f1df1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.206248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887ed95-6ecf-4506-8a70-904d6b4f1df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.208802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 980/1952] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:43 2022] DELETE /identity/v3/users/11e60ca4cf7a471c9113b801c8fd103c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:44.230562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1887ed95-6ecf-4506-8a70-904d6b4f1df1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:44.232473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1887ed95-6ecf-4506-8a70-904d6b4f1df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:44.272927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.274624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:44.312323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:44.359921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 973/1953] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:44.361339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887ed95-6ecf-4506-8a70-904d6b4f1df1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.362467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1887ed95-6ecf-4506-8a70-904d6b4f1df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:44.375230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887ed95-6ecf-4506-8a70-904d6b4f1df1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.378379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ce112-3a89-48eb-9a16-4527deccd631 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.379800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.380694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.380977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.381279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] PATH_INFO: `/v3/projects/f02f5ba2d0f34464bef6f6d30830766f` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.382798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f02f5ba2d0f34464bef6f6d30830766f)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.382923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.383774 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.384051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.384342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] PATH_INFO: `/v3/projects/66f40c9370d34e99aee8460199846b36` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.384756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.386232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f40c9370d34e99aee8460199846b36)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.388116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.436171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f02f5ba2d0f34464bef6f6d30830766f', 'request_id': 'req-c714739e-5c0b-43d2-926f-b4cae3edb496'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.437814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f40c9370d34e99aee8460199846b36', 'request_id': 'req-bde2dac2-e519-48dc-af96-530eb404d7b4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.444078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f02f5ba2d0f34464bef6f6d30830766f', 'request_id': 'req-c714739e-5c0b-43d2-926f-b4cae3edb496'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.444231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f40c9370d34e99aee8460199846b36', 'request_id': 'req-bde2dac2-e519-48dc-af96-530eb404d7b4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.450545 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f02f5ba2d0f34464bef6f6d30830766f', 'request_id': 'req-c714739e-5c0b-43d2-926f-b4cae3edb496'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.452612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f40c9370d34e99aee8460199846b36', 'request_id': 'req-bde2dac2-e519-48dc-af96-530eb404d7b4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.461886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] The token cache is being invalidate because project f02f5ba2d0f34464bef6f6d30830766f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:44.462178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.462846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.462954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] The token cache is being invalidate because project 66f40c9370d34e99aee8460199846b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:44.463223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.463871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bde2dac2-e519-48dc-af96-530eb404d7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:44.465100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 974/1954] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:44 2022] DELETE /identity/v3/projects/f02f5ba2d0f34464bef6f6d30830766f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:44.465641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 981/1955] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:44 2022] DELETE /identity/v3/projects/66f40c9370d34e99aee8460199846b36 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:44.466002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.469104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:44.471440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e55155-a924-43df-9c63-f4f2250743d1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.471738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e55155-a924-43df-9c63-f4f2250743d1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.472021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e55155-a924-43df-9c63-f4f2250743d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.498618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c0e55155-a924-43df-9c63-f4f2250743d1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:44.499972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c0e55155-a924-43df-9c63-f4f2250743d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:44.530098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714739e-5c0b-43d2-926f-b4cae3edb496 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.565507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 982/1956] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:44.566912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbea7a0-0018-4967-95bb-e4bc6fce04a3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.567206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbea7a0-0018-4967-95bb-e4bc6fce04a3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.567489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbea7a0-0018-4967-95bb-e4bc6fce04a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.579020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:44.581877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98606-e393-4069-a1a9-8cc5ef615612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.582548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98606-e393-4069-a1a9-8cc5ef615612 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.582834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98606-e393-4069-a1a9-8cc5ef615612 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.583114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98606-e393-4069-a1a9-8cc5ef615612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.585079 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eff98606-e393-4069-a1a9-8cc5ef615612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:44.593484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5fbea7a0-0018-4967-95bb-e4bc6fce04a3 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:44.595110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5fbea7a0-0018-4967-95bb-e4bc6fce04a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:44.598902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98606-e393-4069-a1a9-8cc5ef615612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.600359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98606-e393-4069-a1a9-8cc5ef615612 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.603707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 983/1957] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:44.605282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5aa2-db8d-4303-926f-0742ada0b248 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.605712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5aa2-db8d-4303-926f-0742ada0b248 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.606000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5aa2-db8d-4303-926f-0742ada0b248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.611362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 975/1958] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:44.612779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a294b8-8679-458f-ad68-a0c3bbadba20 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.613103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a294b8-8679-458f-ad68-a0c3bbadba20 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.613386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a294b8-8679-458f-ad68-a0c3bbadba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.624853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-fb4b5aa2-db8d-4303-926f-0742ada0b248 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:44.626074 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fb4b5aa2-db8d-4303-926f-0742ada0b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:44.631796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-16a294b8-8679-458f-ad68-a0c3bbadba20 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:44.633276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-16a294b8-8679-458f-ad68-a0c3bbadba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:44.635700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 984/1959] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:44.641174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5aa2-db8d-4303-926f-0742ada0b248 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.646843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 976/1960] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:44.648437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a294b8-8679-458f-ad68-a0c3bbadba20 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.652789 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5aa2-db8d-4303-926f-0742ada0b248 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.658576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919e28a-ef15-4675-98f9-4a21e09800b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.659492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919e28a-ef15-4675-98f9-4a21e09800b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.659673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a294b8-8679-458f-ad68-a0c3bbadba20 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.659776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919e28a-ef15-4675-98f9-4a21e09800b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.660049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919e28a-ef15-4675-98f9-4a21e09800b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.661201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919e28a-ef15-4675-98f9-4a21e09800b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:44.661517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919e28a-ef15-4675-98f9-4a21e09800b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.663140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919e28a-ef15-4675-98f9-4a21e09800b9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.665041 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a807d30-1459-42e2-a3f8-04bcfa5591a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.667676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807d30-1459-42e2-a3f8-04bcfa5591a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.668032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807d30-1459-42e2-a3f8-04bcfa5591a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.668344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807d30-1459-42e2-a3f8-04bcfa5591a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.669376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a807d30-1459-42e2-a3f8-04bcfa5591a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:44.669601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a807d30-1459-42e2-a3f8-04bcfa5591a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.671775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a807d30-1459-42e2-a3f8-04bcfa5591a1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.674941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:44.679249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 985/1961] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:44.685573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919e28a-ef15-4675-98f9-4a21e09800b9 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.694036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 977/1962] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:44.695849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a807d30-1459-42e2-a3f8-04bcfa5591a1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.699189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919e28a-ef15-4675-98f9-4a21e09800b9 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.710774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a807d30-1459-42e2-a3f8-04bcfa5591a1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.713908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab2caa-8359-414f-9ec7-5861d4d0ba6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.714631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab2caa-8359-414f-9ec7-5861d4d0ba6e admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.714913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab2caa-8359-414f-9ec7-5861d4d0ba6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.715203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab2caa-8359-414f-9ec7-5861d4d0ba6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.715920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab2caa-8359-414f-9ec7-5861d4d0ba6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.717896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab2caa-8359-414f-9ec7-5861d4d0ba6e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.718201 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea7c38-3509-4c6a-99d0-1cd8f75ad41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.718872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea7c38-3509-4c6a-99d0-1cd8f75ad41d admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.719212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea7c38-3509-4c6a-99d0-1cd8f75ad41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.719520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea7c38-3509-4c6a-99d0-1cd8f75ad41d admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.720218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea7c38-3509-4c6a-99d0-1cd8f75ad41d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.722196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea7c38-3509-4c6a-99d0-1cd8f75ad41d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.751601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 986/1963] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:44 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:34:44.753520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 978/1964] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:44 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:44.755777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab2caa-8359-414f-9ec7-5861d4d0ba6e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.757529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea7c38-3509-4c6a-99d0-1cd8f75ad41d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.847888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea7c38-3509-4c6a-99d0-1cd8f75ad41d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.854676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9de581-46b3-4ef6-8cdb-0980582068a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.855463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9de581-46b3-4ef6-8cdb-0980582068a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.855828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9de581-46b3-4ef6-8cdb-0980582068a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.856348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9de581-46b3-4ef6-8cdb-0980582068a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.857388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9de581-46b3-4ef6-8cdb-0980582068a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.861766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9de581-46b3-4ef6-8cdb-0980582068a7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.888681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab2caa-8359-414f-9ec7-5861d4d0ba6e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.896113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f866410-914d-4f3f-b20e-ffad7dbed815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.897179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f866410-914d-4f3f-b20e-ffad7dbed815 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.897649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f866410-914d-4f3f-b20e-ffad7dbed815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.898093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f866410-914d-4f3f-b20e-ffad7dbed815 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.899043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f866410-914d-4f3f-b20e-ffad7dbed815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.900364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 979/1965] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:44 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:44.903686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f866410-914d-4f3f-b20e-ffad7dbed815 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.906599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9de581-46b3-4ef6-8cdb-0980582068a7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.934830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9de581-46b3-4ef6-8cdb-0980582068a7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.936015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 987/1966] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:44 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:44.937237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f866410-914d-4f3f-b20e-ffad7dbed815 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.938028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a94d5d-88f0-46a7-9610-2efa84dc812b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.939165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a94d5d-88f0-46a7-9610-2efa84dc812b admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.939165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a94d5d-88f0-46a7-9610-2efa84dc812b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.939372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a94d5d-88f0-46a7-9610-2efa84dc812b admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.941092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a94d5d-88f0-46a7-9610-2efa84dc812b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.943657 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a94d5d-88f0-46a7-9610-2efa84dc812b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.955368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f866410-914d-4f3f-b20e-ffad7dbed815 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.966710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a54d4-87a8-44b8-8c6c-f055e29892a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.967465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a54d4-87a8-44b8-8c6c-f055e29892a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.967790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a54d4-87a8-44b8-8c6c-f055e29892a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.968073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a54d4-87a8-44b8-8c6c-f055e29892a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.968941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a54d4-87a8-44b8-8c6c-f055e29892a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.969609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 980/1967] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:44.973711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a94d5d-88f0-46a7-9610-2efa84dc812b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.974347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a54d4-87a8-44b8-8c6c-f055e29892a7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:44.986555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a94d5d-88f0-46a7-9610-2efa84dc812b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:44.991655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 988/1968] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:44.991806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53ca64-1348-4848-ac5a-418de4472e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:44.992506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53ca64-1348-4848-ac5a-418de4472e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:44.993432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a54d4-87a8-44b8-8c6c-f055e29892a7 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:44.993572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53ca64-1348-4848-ac5a-418de4472e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:44.993738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53ca64-1348-4848-ac5a-418de4472e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:44.994779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53ca64-1348-4848-ac5a-418de4472e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:44.996503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53ca64-1348-4848-ac5a-418de4472e47 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:45.008456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a54d4-87a8-44b8-8c6c-f055e29892a7 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:45.009438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 981/1969] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:45.014460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53ca64-1348-4848-ac5a-418de4472e47 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:45.025848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53ca64-1348-4848-ac5a-418de4472e47 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:45.027144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35dc9b-8e59-4cf8-98dc-7afe5ac939e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:45.027906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35dc9b-8e59-4cf8-98dc-7afe5ac939e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:45.028201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35dc9b-8e59-4cf8-98dc-7afe5ac939e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:45.028529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35dc9b-8e59-4cf8-98dc-7afe5ac939e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:45.029260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35dc9b-8e59-4cf8-98dc-7afe5ac939e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:45.030910 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35dc9b-8e59-4cf8-98dc-7afe5ac939e8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:45.035169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a855-bcfb-4138-b11d-33c3bfa9f3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:45.036561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a855-bcfb-4138-b11d-33c3bfa9f3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:45.036561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a855-bcfb-4138-b11d-33c3bfa9f3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:45.036790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a855-bcfb-4138-b11d-33c3bfa9f3d4 admin admin] PATH_INFO: `/v3/projects/7da55d6136d549748e95617caa09da14/users/5afe7402a6fb42748b77ba01853b4b54/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:45.052545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 989/1970] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:45.053869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35dc9b-8e59-4cf8-98dc-7afe5ac939e8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:45.054012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a855-bcfb-4138-b11d-33c3bfa9f3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7da55d6136d549748e95617caa09da14, user_id=5afe7402a6fb42748b77ba01853b4b54, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:45.057324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a855-bcfb-4138-b11d-33c3bfa9f3d4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:45.069649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35dc9b-8e59-4cf8-98dc-7afe5ac939e8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:45.080386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 982/1971] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:45 2022] PUT /identity/v3/projects/7da55d6136d549748e95617caa09da14/users/5afe7402a6fb42748b77ba01853b4b54/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:45.091563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe747f3-67ef-4c3d-8602-1b530cab755d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:45.093034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe747f3-67ef-4c3d-8602-1b530cab755d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:45.093410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe747f3-67ef-4c3d-8602-1b530cab755d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:45.132849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-bbe747f3-67ef-4c3d-8602-1b530cab755d None None] MFA Rules not processed for user `5afe7402a6fb42748b77ba01853b4b54`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:45.134256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bbe747f3-67ef-4c3d-8602-1b530cab755d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:45.177159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd7d6b-9a36-4793-bb37-61311041acfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:45.178638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7d6b-9a36-4793-bb37-61311041acfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:45.179116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7d6b-9a36-4793-bb37-61311041acfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:45.179687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7d6b-9a36-4793-bb37-61311041acfb admin admin] PATH_INFO: `/v3/projects/00c61e549718467bb3fa9f3db9014f0a/users/9ee1da8f3a9f42b1bda7c06eb13d1d09/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:45.223287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd7d6b-9a36-4793-bb37-61311041acfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=00c61e549718467bb3fa9f3db9014f0a, user_id=9ee1da8f3a9f42b1bda7c06eb13d1d09, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:45.230480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd7d6b-9a36-4793-bb37-61311041acfb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:45.248582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 983/1972] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:45.250032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe747f3-67ef-4c3d-8602-1b530cab755d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:45.261699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bbe747f3-67ef-4c3d-8602-1b530cab755d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:45.274002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 990/1973] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:45 2022] PUT /identity/v3/projects/00c61e549718467bb3fa9f3db9014f0a/users/9ee1da8f3a9f42b1bda7c06eb13d1d09/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:45.274002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd7d6b-9a36-4793-bb37-61311041acfb admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:45.276656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f9dd7d6b-9a36-4793-bb37-61311041acfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:45.292990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe747f3-67ef-4c3d-8602-1b530cab755d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:45.298647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:45.316572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:45.477134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:45.478221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:45.478500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:45.481080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] PATH_INFO: `/v3/projects/2bbd90390a0242dd8be8b1bbbabcaf2a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:45.491094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bbd90390a0242dd8be8b1bbbabcaf2a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:45.496891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:45.500013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd7d6b-9a36-4793-bb37-61311041acfb admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:45.508993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd2603-594b-431a-9960-e06dcc74cee4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e07c59babad479cadbf74f15ee5eea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:45.509764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd2603-594b-431a-9960-e06dcc74cee4 service glance] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:45.512632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd2603-594b-431a-9960-e06dcc74cee4 service glance] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:45.512916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd2603-594b-431a-9960-e06dcc74cee4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:45.591317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bbd90390a0242dd8be8b1bbbabcaf2a', 'request_id': 'req-1e524368-6d88-44ee-9732-72666abccef4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:45.603702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bbd90390a0242dd8be8b1bbbabcaf2a', 'request_id': 'req-1e524368-6d88-44ee-9732-72666abccef4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:45.621919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bbd90390a0242dd8be8b1bbbabcaf2a', 'request_id': 'req-1e524368-6d88-44ee-9732-72666abccef4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:45.655865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] The token cache is being invalidate because project 2bbd90390a0242dd8be8b1bbbabcaf2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:45.656320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:45.657033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1e524368-6d88-44ee-9732-72666abccef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:45.661970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 984/1974] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:45 2022] DELETE /identity/v3/projects/2bbd90390a0242dd8be8b1bbbabcaf2a => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:45.661970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47af1-3fcd-4d75-a7e5-d7b9966d89c5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:45.661970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47af1-3fcd-4d75-a7e5-d7b9966d89c5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:45.661970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47af1-3fcd-4d75-a7e5-d7b9966d89c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:45.663614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd2603-594b-431a-9960-e06dcc74cee4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:45.669284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd2603-594b-431a-9960-e06dcc74cee4 service glance] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:45.694999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 991/1975] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:45.696939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97166f0-ccc9-4437-88d0-2e24be4c9a85 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:45.698322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97166f0-ccc9-4437-88d0-2e24be4c9a85 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:45.699322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97166f0-ccc9-4437-88d0-2e24be4c9a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:45.703180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-15b47af1-3fcd-4d75-a7e5-d7b9966d89c5 None None] MFA Rules not processed for user `9ee1da8f3a9f42b1bda7c06eb13d1d09`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:45.707206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-15b47af1-3fcd-4d75-a7e5-d7b9966d89c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:45.743710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a97166f0-ccc9-4437-88d0-2e24be4c9a85 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:45.745360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a97166f0-ccc9-4437-88d0-2e24be4c9a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:45.870770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 985/1976] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:45.875993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 992/1977] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:45.883741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02b83-7c57-4912-9979-e4916a9a735c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:45.884024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02b83-7c57-4912-9979-e4916a9a735c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:45.884319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02b83-7c57-4912-9979-e4916a9a735c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:45.904920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1ec02b83-7c57-4912-9979-e4916a9a735c None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:45.906260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1ec02b83-7c57-4912-9979-e4916a9a735c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:45.924236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 986/1978] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:45.928799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97166f0-ccc9-4437-88d0-2e24be4c9a85 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:45.949105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97166f0-ccc9-4437-88d0-2e24be4c9a85 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:45.964914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b64d1-abcd-4783-9575-aba0404eaff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:45.966368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b64d1-abcd-4783-9575-aba0404eaff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:45.966656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b64d1-abcd-4783-9575-aba0404eaff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:45.966975 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b64d1-abcd-4783-9575-aba0404eaff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:45.967807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b64d1-abcd-4783-9575-aba0404eaff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:45.968106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b64d1-abcd-4783-9575-aba0404eaff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:45.972495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b64d1-abcd-4783-9575-aba0404eaff5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:45.986700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec02b83-7c57-4912-9979-e4916a9a735c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:45.988176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1ec02b83-7c57-4912-9979-e4916a9a735c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:45.992371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 993/1979] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:45.995124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b64d1-abcd-4783-9575-aba0404eaff5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:46.014249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b64d1-abcd-4783-9575-aba0404eaff5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:46.016827 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:46.025993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11146427-bc16-4f13-be3c-e3a4ee51932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:46.026907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11146427-bc16-4f13-be3c-e3a4ee51932f admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:46.027226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11146427-bc16-4f13-be3c-e3a4ee51932f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:46.027560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11146427-bc16-4f13-be3c-e3a4ee51932f admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:46.029375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11146427-bc16-4f13-be3c-e3a4ee51932f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:46.034266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11146427-bc16-4f13-be3c-e3a4ee51932f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:46.069500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec02b83-7c57-4912-9979-e4916a9a735c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:46.078337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 994/1980] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:45 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:46.079580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11146427-bc16-4f13-be3c-e3a4ee51932f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:46.085624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d1c1d-42f5-43de-a21c-279a69bd693c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:46.086302 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d1c1d-42f5-43de-a21c-279a69bd693c service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:46.086584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d1c1d-42f5-43de-a21c-279a69bd693c service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:46.086861 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d1c1d-42f5-43de-a21c-279a69bd693c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:46.091471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:46.110108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d1c1d-42f5-43de-a21c-279a69bd693c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:46.111705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d1c1d-42f5-43de-a21c-279a69bd693c service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:46.127748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 987/1981] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:46.161758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11146427-bc16-4f13-be3c-e3a4ee51932f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:46.168625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e19f36-cb43-4850-a278-6da75f111a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:46.169447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e19f36-cb43-4850-a278-6da75f111a59 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:46.169774 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e19f36-cb43-4850-a278-6da75f111a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:46.170117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e19f36-cb43-4850-a278-6da75f111a59 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:46.170895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e19f36-cb43-4850-a278-6da75f111a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:46.173490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e19f36-cb43-4850-a278-6da75f111a59 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:46.200037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 995/1982] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:46 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:34:46.204243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d1c1d-42f5-43de-a21c-279a69bd693c service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:46.218859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d1c1d-42f5-43de-a21c-279a69bd693c service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:46.224877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d0ca1-a1c5-4356-ae40-530e8ae84553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:46.226708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d0ca1-a1c5-4356-ae40-530e8ae84553 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:46.227473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d0ca1-a1c5-4356-ae40-530e8ae84553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:46.227765 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d0ca1-a1c5-4356-ae40-530e8ae84553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:46.229778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d0ca1-a1c5-4356-ae40-530e8ae84553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:46.232443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d0ca1-a1c5-4356-ae40-530e8ae84553 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:46.245831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 988/1983] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:46.250048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e19f36-cb43-4850-a278-6da75f111a59 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:46.260913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e19f36-cb43-4850-a278-6da75f111a59 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:46.282178 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa683e35-48da-4afa-a925-41f6bdc30106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:46.282870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa683e35-48da-4afa-a925-41f6bdc30106 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:46.283141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa683e35-48da-4afa-a925-41f6bdc30106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:46.283534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa683e35-48da-4afa-a925-41f6bdc30106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:46.284321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa683e35-48da-4afa-a925-41f6bdc30106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:46.285954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa683e35-48da-4afa-a925-41f6bdc30106 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:46.313646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 996/1984] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:46.318015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d0ca1-a1c5-4356-ae40-530e8ae84553 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:46.350102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d0ca1-a1c5-4356-ae40-530e8ae84553 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:46.364563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9bc77-7ca8-4db2-b89b-9af94d931c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:46.365516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9bc77-7ca8-4db2-b89b-9af94d931c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:46.365850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9bc77-7ca8-4db2-b89b-9af94d931c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:46.366035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9bc77-7ca8-4db2-b89b-9af94d931c29 admin admin] PATH_INFO: `/v3/projects/3c3d76edc85a40b787568964f342c714/users/45dab588197145b08b49977a99f170f8/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:46.402493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9bc77-7ca8-4db2-b89b-9af94d931c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c3d76edc85a40b787568964f342c714, user_id=45dab588197145b08b49977a99f170f8, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:46.413461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9bc77-7ca8-4db2-b89b-9af94d931c29 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:46.470866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 989/1985] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:46 2022] PUT /identity/v3/projects/3c3d76edc85a40b787568964f342c714/users/45dab588197145b08b49977a99f170f8/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:46.472975 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188c6b9-778c-4183-b0f8-a24b13228bb3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:46.473270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188c6b9-778c-4183-b0f8-a24b13228bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:46.473552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188c6b9-778c-4183-b0f8-a24b13228bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:46.553727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d188c6b9-778c-4183-b0f8-a24b13228bb3 None None] MFA Rules not processed for user `45dab588197145b08b49977a99f170f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:46.555946 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d188c6b9-778c-4183-b0f8-a24b13228bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:46.731412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 997/1986] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:46.736103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9bc77-7ca8-4db2-b89b-9af94d931c29 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:46.841574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9bc77-7ca8-4db2-b89b-9af94d931c29 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:46.848590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca28a46-476b-472b-a611-a181121cd640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:46.849415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca28a46-476b-472b-a611-a181121cd640 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:46.849780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca28a46-476b-472b-a611-a181121cd640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:46.850154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca28a46-476b-472b-a611-a181121cd640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:46.850966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca28a46-476b-472b-a611-a181121cd640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:46.853970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca28a46-476b-472b-a611-a181121cd640 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:46.889564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 990/1987] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:46 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:34:46.893352 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188c6b9-778c-4183-b0f8-a24b13228bb3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:46.910049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188c6b9-778c-4183-b0f8-a24b13228bb3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:46.991204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8cdf-27d1-4b88-bb84-70a4316756b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:46.991923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8cdf-27d1-4b88-bb84-70a4316756b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:46.992341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8cdf-27d1-4b88-bb84-70a4316756b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:46.992635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8cdf-27d1-4b88-bb84-70a4316756b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:46.993325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8cdf-27d1-4b88-bb84-70a4316756b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:46.996469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8cdf-27d1-4b88-bb84-70a4316756b3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:47.018670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 998/1988] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:46 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:34:47.022557 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca28a46-476b-472b-a611-a181121cd640 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:47.035639 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca28a46-476b-472b-a611-a181121cd640 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:47.040447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958edbb-e0a4-4638-8a13-9f9c4cec1a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:47.041831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958edbb-e0a4-4638-8a13-9f9c4cec1a07 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:47.042122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958edbb-e0a4-4638-8a13-9f9c4cec1a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:47.042401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958edbb-e0a4-4638-8a13-9f9c4cec1a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:47.043137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958edbb-e0a4-4638-8a13-9f9c4cec1a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:47.044811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958edbb-e0a4-4638-8a13-9f9c4cec1a07 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:47.056332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 991/1989] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:34:47.062833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8cdf-27d1-4b88-bb84-70a4316756b3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:47.067589 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:47.072630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8cdf-27d1-4b88-bb84-70a4316756b3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:47.078777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38062e3e-f739-458f-9daf-c7fbb0c0e641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:47.079792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38062e3e-f739-458f-9daf-c7fbb0c0e641 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:47.080016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38062e3e-f739-458f-9daf-c7fbb0c0e641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:47.080318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38062e3e-f739-458f-9daf-c7fbb0c0e641 admin admin] PATH_INFO: `/v3/projects/6668f7226a8247c1804cc5b82f8f688c/users/93c8e0be60ea4ae992fed46d4c6d60e1/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:47.099122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38062e3e-f739-458f-9daf-c7fbb0c0e641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6668f7226a8247c1804cc5b82f8f688c, user_id=93c8e0be60ea4ae992fed46d4c6d60e1, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:47.101818 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38062e3e-f739-458f-9daf-c7fbb0c0e641 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:47.120111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 999/1990] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:47 2022] PUT /identity/v3/projects/6668f7226a8247c1804cc5b82f8f688c/users/93c8e0be60ea4ae992fed46d4c6d60e1/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:47.124085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958edbb-e0a4-4638-8a13-9f9c4cec1a07 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:47.134036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958edbb-e0a4-4638-8a13-9f9c4cec1a07 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:47.194865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baca157-b7c4-4937-b052-0f536797bcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:47.195499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baca157-b7c4-4937-b052-0f536797bcbd admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:47.195780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baca157-b7c4-4937-b052-0f536797bcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:47.196075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baca157-b7c4-4937-b052-0f536797bcbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:47.196875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baca157-b7c4-4937-b052-0f536797bcbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:47.199324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baca157-b7c4-4937-b052-0f536797bcbd admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:47.201748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:47.211635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 992/1991] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:47.215817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38062e3e-f739-458f-9daf-c7fbb0c0e641 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:47.225983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38062e3e-f739-458f-9daf-c7fbb0c0e641 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:47.231602 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9052e8-0844-4ca5-b89d-0b7c06e1acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:47.232519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9052e8-0844-4ca5-b89d-0b7c06e1acc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:47.232817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9052e8-0844-4ca5-b89d-0b7c06e1acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:47.233097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9052e8-0844-4ca5-b89d-0b7c06e1acc1 admin admin] PATH_INFO: `/v3/projects/6668f7226a8247c1804cc5b82f8f688c/users/93c8e0be60ea4ae992fed46d4c6d60e1/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:47.235071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9052e8-0844-4ca5-b89d-0b7c06e1acc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6668f7226a8247c1804cc5b82f8f688c, user_id=93c8e0be60ea4ae992fed46d4c6d60e1, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:47.237802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9052e8-0844-4ca5-b89d-0b7c06e1acc1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:47.250091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1000/1992] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:47 2022] PUT /identity/v3/projects/6668f7226a8247c1804cc5b82f8f688c/users/93c8e0be60ea4ae992fed46d4c6d60e1/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:47.261236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5d4c4-179e-49fd-819f-59b3d39b9293 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:47.261535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5d4c4-179e-49fd-819f-59b3d39b9293 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:47.261824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5d4c4-179e-49fd-819f-59b3d39b9293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:47.288064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-10a5d4c4-179e-49fd-819f-59b3d39b9293 None None] MFA Rules not processed for user `93c8e0be60ea4ae992fed46d4c6d60e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:47.289322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-10a5d4c4-179e-49fd-819f-59b3d39b9293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:47.357721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 993/1993] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:47.369396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9052e8-0844-4ca5-b89d-0b7c06e1acc1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:47.371480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ba9052e8-0844-4ca5-b89d-0b7c06e1acc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:47.452599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9052e8-0844-4ca5-b89d-0b7c06e1acc1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:47.456828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b10f9-2903-4938-8f97-75646c8020b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:47.457454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b10f9-2903-4938-8f97-75646c8020b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:47.457767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b10f9-2903-4938-8f97-75646c8020b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:47.458111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b10f9-2903-4938-8f97-75646c8020b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:47.521751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b10f9-2903-4938-8f97-75646c8020b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:47.523214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b10f9-2903-4938-8f97-75646c8020b6 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:47.533178 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1001/1994] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:48.096798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5d4c4-179e-49fd-819f-59b3d39b9293 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:48.098102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-10a5d4c4-179e-49fd-819f-59b3d39b9293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:48.176712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5d4c4-179e-49fd-819f-59b3d39b9293 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:48.185666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:48.186857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:48.186970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:48.187166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] PATH_INFO: `/v3/users/c8aaa561314447c7a1ef0a159cc015db` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:48.189072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8aaa561314447c7a1ef0a159cc015db)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:48.191028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:48.262951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8aaa561314447c7a1ef0a159cc015db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:48.263823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8aaa561314447c7a1ef0a159cc015db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:48.264482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8aaa561314447c7a1ef0a159cc015db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:48.274464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8aaa561314447c7a1ef0a159cc015db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:48.288577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8aaa561314447c7a1ef0a159cc015db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:48.310350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 994/1995] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:48 2022] DELETE /identity/v3/users/c8aaa561314447c7a1ef0a159cc015db => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:48.317389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fd1d7-d8da-4f0b-83a6-648f78568195 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:48.317856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fd1d7-d8da-4f0b-83a6-648f78568195 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:48.318018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fd1d7-d8da-4f0b-83a6-648f78568195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:48.369040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ee4fd1d7-d8da-4f0b-83a6-648f78568195 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:48.370938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ee4fd1d7-d8da-4f0b-83a6-648f78568195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:48.473166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1002/1996] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:48.493537 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:48.496095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:48.513061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:48.598090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3ebdf-43c6-48f6-9d53-139ef0cb2047 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:48.603019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce62b0f-e7cf-42a6-a208-f18a05c10f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:48.604750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce62b0f-e7cf-42a6-a208-f18a05c10f47 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:48.604750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce62b0f-e7cf-42a6-a208-f18a05c10f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:48.605087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce62b0f-e7cf-42a6-a208-f18a05c10f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:48.623124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce62b0f-e7cf-42a6-a208-f18a05c10f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:48.625432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce62b0f-e7cf-42a6-a208-f18a05c10f47 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:48.639641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 995/1997] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:48.952653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fd1d7-d8da-4f0b-83a6-648f78568195 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:48.954453 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ee4fd1d7-d8da-4f0b-83a6-648f78568195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:48.969645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fd1d7-d8da-4f0b-83a6-648f78568195 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:48.975656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:48.977200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:48.977686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:48.978165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] PATH_INFO: `/v3/users/9ee1da8f3a9f42b1bda7c06eb13d1d09` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:48.980323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee1da8f3a9f42b1bda7c06eb13d1d09)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:48.993507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:49.037284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:49.102981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee1da8f3a9f42b1bda7c06eb13d1d09'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:49.113969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee1da8f3a9f42b1bda7c06eb13d1d09'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:49.115495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee1da8f3a9f42b1bda7c06eb13d1d09'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:49.139055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee1da8f3a9f42b1bda7c06eb13d1d09'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:49.152838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee1da8f3a9f42b1bda7c06eb13d1d09'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:49.163550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1003/1998] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:48 2022] DELETE /identity/v3/users/9ee1da8f3a9f42b1bda7c06eb13d1d09 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:49.172705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8da9e-3f54-44cc-910b-492c60d7518d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:49.173226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8da9e-3f54-44cc-910b-492c60d7518d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:49.173321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8da9e-3f54-44cc-910b-492c60d7518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:49.192811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3cc8da9e-3f54-44cc-910b-492c60d7518d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:49.194653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3cc8da9e-3f54-44cc-910b-492c60d7518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:49.271646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 996/1999] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:49.283624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:49.285223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:49.355612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116fdeb-9fe9-452e-89b1-07111c35e6b1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:49.359667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0125d-32f2-47f4-8072-e8076c917296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:49.360339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0125d-32f2-47f4-8072-e8076c917296 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:49.360628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0125d-32f2-47f4-8072-e8076c917296 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:49.360904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0125d-32f2-47f4-8072-e8076c917296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:49.370519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0125d-32f2-47f4-8072-e8076c917296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:49.372131 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0125d-32f2-47f4-8072-e8076c917296 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:49.384015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1004/2000] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:49.779102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8da9e-3f54-44cc-910b-492c60d7518d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:49.780602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3cc8da9e-3f54-44cc-910b-492c60d7518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:49.801357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8da9e-3f54-44cc-910b-492c60d7518d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:49.809568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:49.810463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:49.810776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:49.826024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] PATH_INFO: `/v3/projects/00c61e549718467bb3fa9f3db9014f0a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:49.828479 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00c61e549718467bb3fa9f3db9014f0a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:49.831708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:49.835561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0125d-32f2-47f4-8072-e8076c917296 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:49.837815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-aaa0125d-32f2-47f4-8072-e8076c917296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:49.858286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0125d-32f2-47f4-8072-e8076c917296 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:49.866977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:49.868023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:49.868473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:49.868857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] PATH_INFO: `/v3/projects/51c276db6e4d4a2da6c201a02b09d800` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:49.871266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c276db6e4d4a2da6c201a02b09d800)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:49.873535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:49.947417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00c61e549718467bb3fa9f3db9014f0a', 'request_id': 'req-7e67de15-9e97-4a47-90d0-43ef08e59f87'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:49.959218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00c61e549718467bb3fa9f3db9014f0a', 'request_id': 'req-7e67de15-9e97-4a47-90d0-43ef08e59f87'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:49.967089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00c61e549718467bb3fa9f3db9014f0a', 'request_id': 'req-7e67de15-9e97-4a47-90d0-43ef08e59f87'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:49.981139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] The token cache is being invalidate because project 00c61e549718467bb3fa9f3db9014f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:49.981567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:49.982489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7e67de15-9e97-4a47-90d0-43ef08e59f87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:49.988139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 997/2001] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:49 2022] DELETE /identity/v3/projects/00c61e549718467bb3fa9f3db9014f0a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:49.989755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521173a-4474-4b37-bd16-313794bcb2b4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:49.990154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521173a-4474-4b37-bd16-313794bcb2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:49.990616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521173a-4474-4b37-bd16-313794bcb2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.007404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c276db6e4d4a2da6c201a02b09d800', 'request_id': 'req-27e268de-edc9-461f-9e85-50cf51fb3569'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.012556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e521173a-4474-4b37-bd16-313794bcb2b4 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:50.014223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e521173a-4474-4b37-bd16-313794bcb2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:50.015218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c276db6e4d4a2da6c201a02b09d800', 'request_id': 'req-27e268de-edc9-461f-9e85-50cf51fb3569'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.027527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c276db6e4d4a2da6c201a02b09d800', 'request_id': 'req-27e268de-edc9-461f-9e85-50cf51fb3569'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.039871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] The token cache is being invalidate because project 51c276db6e4d4a2da6c201a02b09d800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:50.040394 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.041652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.043250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1005/2002] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:49 2022] DELETE /identity/v3/projects/51c276db6e4d4a2da6c201a02b09d800 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:50.047209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:50.048665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:50.068428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:50.089888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:50.139629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 998/2003] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:50.148768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3415f0-89a0-4449-8804-2f4ae9e5ddf6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:50.149067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3415f0-89a0-4449-8804-2f4ae9e5ddf6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:50.149354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3415f0-89a0-4449-8804-2f4ae9e5ddf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.176955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e268de-edc9-461f-9e85-50cf51fb3569 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:50.181576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5a3415f0-89a0-4449-8804-2f4ae9e5ddf6 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:50.183847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5a3415f0-89a0-4449-8804-2f4ae9e5ddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:50.188643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:50.189535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:50.189814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:50.190086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] PATH_INFO: `/v3/users/a855023c40af4e31941ee1c24531f825` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.191358 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a855023c40af4e31941ee1c24531f825)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:50.193507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:50.196497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 999/2004] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:50.200956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3415f0-89a0-4449-8804-2f4ae9e5ddf6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:50.216531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3415f0-89a0-4449-8804-2f4ae9e5ddf6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:50.221803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c62c63-a1cd-4795-900e-7d8caf50fe56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:50.222555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c62c63-a1cd-4795-900e-7d8caf50fe56 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:50.223026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c62c63-a1cd-4795-900e-7d8caf50fe56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:50.223505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c62c63-a1cd-4795-900e-7d8caf50fe56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.227045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c62c63-a1cd-4795-900e-7d8caf50fe56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:50.227590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c62c63-a1cd-4795-900e-7d8caf50fe56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:50.232925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c62c63-a1cd-4795-900e-7d8caf50fe56 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:50.256603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1000/2005] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:34:50.257955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c62c63-a1cd-4795-900e-7d8caf50fe56 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:50.272950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c62c63-a1cd-4795-900e-7d8caf50fe56 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:50.274923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a855023c40af4e31941ee1c24531f825'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.279582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980768d-1f1f-4c1b-b0a7-310c1238e9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:50.280268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980768d-1f1f-4c1b-b0a7-310c1238e9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:50.280592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980768d-1f1f-4c1b-b0a7-310c1238e9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:50.280873 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980768d-1f1f-4c1b-b0a7-310c1238e9a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.281628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980768d-1f1f-4c1b-b0a7-310c1238e9a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:50.283604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980768d-1f1f-4c1b-b0a7-310c1238e9a8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:50.289665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a855023c40af4e31941ee1c24531f825'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.292877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a855023c40af4e31941ee1c24531f825'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.319190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a855023c40af4e31941ee1c24531f825'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.319395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1001/2006] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:50 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:50.324967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980768d-1f1f-4c1b-b0a7-310c1238e9a8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:50.334263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a855023c40af4e31941ee1c24531f825'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.348721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1006/2007] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:50 2022] DELETE /identity/v3/users/a855023c40af4e31941ee1c24531f825 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:50.348721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:50.349908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:50.445660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980768d-1f1f-4c1b-b0a7-310c1238e9a8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:50.498538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f1700-72fc-4c06-a2fa-ea79fe5ac14c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:50.514620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:50.515691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:50.516079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:50.516501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] PATH_INFO: `/v3/users/45dab588197145b08b49977a99f170f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.519054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] RBAC: Authorizing `identity:delete_user(user_id=45dab588197145b08b49977a99f170f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:50.521251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:50.562766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9a411-d6d4-44c7-93c5-2d55c2ec5f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:50.564031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9a411-d6d4-44c7-93c5-2d55c2ec5f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:50.564460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9a411-d6d4-44c7-93c5-2d55c2ec5f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:50.564627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9a411-d6d4-44c7-93c5-2d55c2ec5f62 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.565325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9a411-d6d4-44c7-93c5-2d55c2ec5f62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:50.567246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9a411-d6d4-44c7-93c5-2d55c2ec5f62 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:50.599082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:50.621704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1002/2008] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:50 2022] POST /identity/v3/users => generated 292 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:34:50.621959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45dab588197145b08b49977a99f170f8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.625464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9a411-d6d4-44c7-93c5-2d55c2ec5f62 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:50.629243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-07d9a411-d6d4-44c7-93c5-2d55c2ec5f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:50.648768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45dab588197145b08b49977a99f170f8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.650506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45dab588197145b08b49977a99f170f8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.658701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9a411-d6d4-44c7-93c5-2d55c2ec5f62 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:50.672674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079fc36-5909-4c3f-a307-de505b7dd29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:50.673384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079fc36-5909-4c3f-a307-de505b7dd29e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:50.673652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079fc36-5909-4c3f-a307-de505b7dd29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:50.674679 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079fc36-5909-4c3f-a307-de505b7dd29e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.675430 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079fc36-5909-4c3f-a307-de505b7dd29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:50.680599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079fc36-5909-4c3f-a307-de505b7dd29e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:50.682551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45dab588197145b08b49977a99f170f8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.696385 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7370cef5-5df1-4bd7-823f-db77b156a49a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45dab588197145b08b49977a99f170f8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:50.707036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1003/2009] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:34:50.714988 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079fc36-5909-4c3f-a307-de505b7dd29e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:50.719218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d079fc36-5909-4c3f-a307-de505b7dd29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:50.722084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1007/2010] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:50 2022] DELETE /identity/v3/users/45dab588197145b08b49977a99f170f8 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:50.726748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac09d9c-d144-4005-8f87-211bd15df824 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:50.727203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac09d9c-d144-4005-8f87-211bd15df824 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:50.727580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac09d9c-d144-4005-8f87-211bd15df824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.742879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079fc36-5909-4c3f-a307-de505b7dd29e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:50.753940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0ac09d9c-d144-4005-8f87-211bd15df824 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:50.755630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0ac09d9c-d144-4005-8f87-211bd15df824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:50.759084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:50.846238 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed039005-5548-4664-9179-e1233f7fe9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:50.848928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed039005-5548-4664-9179-e1233f7fe9db admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:50.848928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed039005-5548-4664-9179-e1233f7fe9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:50.851181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed039005-5548-4664-9179-e1233f7fe9db admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.853099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed039005-5548-4664-9179-e1233f7fe9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:50.855579 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed039005-5548-4664-9179-e1233f7fe9db admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:50.860461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1008/2011] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:50.861472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac09d9c-d144-4005-8f87-211bd15df824 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:50.863755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0ac09d9c-d144-4005-8f87-211bd15df824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:50.873159 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1004/2012] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:50.878085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed039005-5548-4664-9179-e1233f7fe9db admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:50.890304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed039005-5548-4664-9179-e1233f7fe9db admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:50.896929 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0de93-8763-48fa-a86d-28e514587631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:50.898263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0de93-8763-48fa-a86d-28e514587631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:50.898801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0de93-8763-48fa-a86d-28e514587631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:50.899366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0de93-8763-48fa-a86d-28e514587631 admin admin] PATH_INFO: `/v3/projects/e47e6891d6044e2caf067cad4309c57f/users/ac4cfa3ccdab43c3adafcef62e62365f/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:50.931534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0de93-8763-48fa-a86d-28e514587631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47e6891d6044e2caf067cad4309c57f, user_id=ac4cfa3ccdab43c3adafcef62e62365f, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:50.934371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0de93-8763-48fa-a86d-28e514587631 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:50.957792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1005/2013] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:50 2022] PUT /identity/v3/projects/e47e6891d6044e2caf067cad4309c57f/users/ac4cfa3ccdab43c3adafcef62e62365f/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:50.959132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0de93-8763-48fa-a86d-28e514587631 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:50.961730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c3e0de93-8763-48fa-a86d-28e514587631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:50.983994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac09d9c-d144-4005-8f87-211bd15df824 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:51.045938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d170a77-635f-4bae-abe4-16449f1c736a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:51.046667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d170a77-635f-4bae-abe4-16449f1c736a service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.047049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d170a77-635f-4bae-abe4-16449f1c736a service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.047158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d170a77-635f-4bae-abe4-16449f1c736a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.080636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0de93-8763-48fa-a86d-28e514587631 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:51.085793 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:51.086695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.087021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.087299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] PATH_INFO: `/v3/projects/45c780ff6f8348ddb4dd62fa488202ad` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.088611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c780ff6f8348ddb4dd62fa488202ad)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:51.090699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:51.095776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:51.146732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d170a77-635f-4bae-abe4-16449f1c736a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:51.148236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d170a77-635f-4bae-abe4-16449f1c736a service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:51.154968 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c780ff6f8348ddb4dd62fa488202ad', 'request_id': 'req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:51.160664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1009/2014] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:51.162033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013bfcc-5c5a-417a-bbcf-7e8f47316984 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.162322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013bfcc-5c5a-417a-bbcf-7e8f47316984 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.162599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013bfcc-5c5a-417a-bbcf-7e8f47316984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.176647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c780ff6f8348ddb4dd62fa488202ad', 'request_id': 'req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:51.186791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c780ff6f8348ddb4dd62fa488202ad', 'request_id': 'req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:51.195255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5013bfcc-5c5a-417a-bbcf-7e8f47316984 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:51.195485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5013bfcc-5c5a-417a-bbcf-7e8f47316984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:51.206953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] The token cache is being invalidate because project 45c780ff6f8348ddb4dd62fa488202ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:51.207343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:51.208314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-934dbdf8-f7c2-4d8a-acc8-4569d91fbd82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:51.210291 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1006/2015] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:50 2022] DELETE /identity/v3/projects/45c780ff6f8348ddb4dd62fa488202ad => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:51.216875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47ede9-23a6-4a9b-b2d1-17e07fe5b946 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.216875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47ede9-23a6-4a9b-b2d1-17e07fe5b946 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.217143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47ede9-23a6-4a9b-b2d1-17e07fe5b946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.242522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5e47ede9-23a6-4a9b-b2d1-17e07fe5b946 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:51.245753 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5e47ede9-23a6-4a9b-b2d1-17e07fe5b946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:51.261855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:51.317407 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1010/2016] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:51.325329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013bfcc-5c5a-417a-bbcf-7e8f47316984 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:51.327908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5013bfcc-5c5a-417a-bbcf-7e8f47316984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:51.335762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1007/2017] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:51.345485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e8b7-3f11-4808-9144-5aa06886eb73 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.357498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e8b7-3f11-4808-9144-5aa06886eb73 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.360128 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e8b7-3f11-4808-9144-5aa06886eb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.386580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ad08e8b7-3f11-4808-9144-5aa06886eb73 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:51.388080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ad08e8b7-3f11-4808-9144-5aa06886eb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:51.407062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1008/2018] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:51.408265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08e8b7-3f11-4808-9144-5aa06886eb73 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:51.414582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013bfcc-5c5a-417a-bbcf-7e8f47316984 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:51.419152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08e8b7-3f11-4808-9144-5aa06886eb73 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:51.419646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfa9d8-c3c3-4368-85a4-b5f3bc831115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:51.420520 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfa9d8-c3c3-4368-85a4-b5f3bc831115 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.420726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfa9d8-c3c3-4368-85a4-b5f3bc831115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.421102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfa9d8-c3c3-4368-85a4-b5f3bc831115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.425258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f195ef-fb0d-4819-9ef1-6e3acb010922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:51.425804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f195ef-fb0d-4819-9ef1-6e3acb010922 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.426138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f195ef-fb0d-4819-9ef1-6e3acb010922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.426434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f195ef-fb0d-4819-9ef1-6e3acb010922 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.427204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f195ef-fb0d-4819-9ef1-6e3acb010922 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:51.427686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f195ef-fb0d-4819-9ef1-6e3acb010922 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:51.429192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f195ef-fb0d-4819-9ef1-6e3acb010922 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:51.434315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfa9d8-c3c3-4368-85a4-b5f3bc831115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:51.435888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfa9d8-c3c3-4368-85a4-b5f3bc831115 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:51.445406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1009/2019] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:51.446571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1011/2020] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:51.450988 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f195ef-fb0d-4819-9ef1-6e3acb010922 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:51.463153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f195ef-fb0d-4819-9ef1-6e3acb010922 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:51.468561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a33e5-03ba-41b7-8ec2-1b3e0d90a310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:51.469248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a33e5-03ba-41b7-8ec2-1b3e0d90a310 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.469757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a33e5-03ba-41b7-8ec2-1b3e0d90a310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.470328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a33e5-03ba-41b7-8ec2-1b3e0d90a310 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.471148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a33e5-03ba-41b7-8ec2-1b3e0d90a310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:51.473137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a33e5-03ba-41b7-8ec2-1b3e0d90a310 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:51.507993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1010/2021] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:51 2022] POST /identity/v3/projects => generated 393 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:51.513870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfa9d8-c3c3-4368-85a4-b5f3bc831115 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:51.526239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfa9d8-c3c3-4368-85a4-b5f3bc831115 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:51.583774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a33e5-03ba-41b7-8ec2-1b3e0d90a310 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:51.585490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8b4a33e5-03ba-41b7-8ec2-1b3e0d90a310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:51.652088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd18c9a-fda9-4a2b-a06b-313b003a4070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:51.652881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18c9a-fda9-4a2b-a06b-313b003a4070 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.653262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18c9a-fda9-4a2b-a06b-313b003a4070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.653641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18c9a-fda9-4a2b-a06b-313b003a4070 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.654324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd18c9a-fda9-4a2b-a06b-313b003a4070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:51.662291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd18c9a-fda9-4a2b-a06b-313b003a4070 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:51.683561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:51.701312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a33e5-03ba-41b7-8ec2-1b3e0d90a310 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:51.705065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1012/2022] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:51 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:51.709319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd18c9a-fda9-4a2b-a06b-313b003a4070 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:51.716442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:51.717513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.718285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.719796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] PATH_INFO: `/v3/projects/3c3d76edc85a40b787568964f342c714` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.721686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c3d76edc85a40b787568964f342c714)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:51.726800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:51.740467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd18c9a-fda9-4a2b-a06b-313b003a4070 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:51.761579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a4e7e-d89c-48b3-9b6e-10f28c0ef2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:51.762593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a4e7e-d89c-48b3-9b6e-10f28c0ef2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:51.762593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a4e7e-d89c-48b3-9b6e-10f28c0ef2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:51.762873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a4e7e-d89c-48b3-9b6e-10f28c0ef2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:51.763681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a4e7e-d89c-48b3-9b6e-10f28c0ef2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:51.765469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a4e7e-d89c-48b3-9b6e-10f28c0ef2e5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:51.786241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1013/2023] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:51.787595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a4e7e-d89c-48b3-9b6e-10f28c0ef2e5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:51.802402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a4e7e-d89c-48b3-9b6e-10f28c0ef2e5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:51.841345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c3d76edc85a40b787568964f342c714', 'request_id': 'req-ad930aec-3f4d-43de-b87c-ae69e35506db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:51.877984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c3d76edc85a40b787568964f342c714', 'request_id': 'req-ad930aec-3f4d-43de-b87c-ae69e35506db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:51.899365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c3d76edc85a40b787568964f342c714', 'request_id': 'req-ad930aec-3f4d-43de-b87c-ae69e35506db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:51.941325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] The token cache is being invalidate because project 3c3d76edc85a40b787568964f342c714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:51.942017 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:51.945375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:51.954822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1011/2024] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:51 2022] DELETE /identity/v3/projects/3c3d76edc85a40b787568964f342c714 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:51.955807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:51.964282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:51.977127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:52.020694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56209ded-d2fd-4ddd-aa52-cf1310068d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:52.024253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56209ded-d2fd-4ddd-aa52-cf1310068d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:52.027406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56209ded-d2fd-4ddd-aa52-cf1310068d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:52.027865 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56209ded-d2fd-4ddd-aa52-cf1310068d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:52.029162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56209ded-d2fd-4ddd-aa52-cf1310068d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:52.031027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56209ded-d2fd-4ddd-aa52-cf1310068d73 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:52.062429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1014/2025] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:52.065299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56209ded-d2fd-4ddd-aa52-cf1310068d73 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:52.069182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-56209ded-d2fd-4ddd-aa52-cf1310068d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:52.099238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56209ded-d2fd-4ddd-aa52-cf1310068d73 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:52.106400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad930aec-3f4d-43de-b87c-ae69e35506db admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:52.110879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ec1ef-e4a8-4480-b691-abad3d93c50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:52.112186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:52.113415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:52.113415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:52.113715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] PATH_INFO: `/v3/users/93c8e0be60ea4ae992fed46d4c6d60e1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:52.114097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ec1ef-e4a8-4480-b691-abad3d93c50e admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:52.115019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c8e0be60ea4ae992fed46d4c6d60e1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:52.115383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ec1ef-e4a8-4480-b691-abad3d93c50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:52.116717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ec1ef-e4a8-4480-b691-abad3d93c50e admin admin] PATH_INFO: `/v3/projects/3ab92b909dc04e95938bbcf591700fcd/users/358425a7d080425185241a7690faa0ec/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:52.117992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:52.170898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ec1ef-e4a8-4480-b691-abad3d93c50e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab92b909dc04e95938bbcf591700fcd, user_id=358425a7d080425185241a7690faa0ec, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:52.173701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ec1ef-e4a8-4480-b691-abad3d93c50e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:52.215916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1015/2026] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:52 2022] PUT /identity/v3/projects/3ab92b909dc04e95938bbcf591700fcd/users/358425a7d080425185241a7690faa0ec/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:52.220000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b329-e7e8-4d80-b4fc-0af3bd70656b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:52.220372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b329-e7e8-4d80-b4fc-0af3bd70656b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:52.220660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b329-e7e8-4d80-b4fc-0af3bd70656b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:52.267452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c8e0be60ea4ae992fed46d4c6d60e1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:52.268444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c8e0be60ea4ae992fed46d4c6d60e1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:52.269322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c8e0be60ea4ae992fed46d4c6d60e1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:52.284121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c8e0be60ea4ae992fed46d4c6d60e1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:52.315426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c8e0be60ea4ae992fed46d4c6d60e1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:52.318856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4782b329-e7e8-4d80-b4fc-0af3bd70656b None None] MFA Rules not processed for user `358425a7d080425185241a7690faa0ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:52.320551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4782b329-e7e8-4d80-b4fc-0af3bd70656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:52.384339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1012/2027] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:51 2022] DELETE /identity/v3/users/93c8e0be60ea4ae992fed46d4c6d60e1 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:52.474414 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1016/2028] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:52.480170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:52.481910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:52.601040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4ce6f-3f9e-45cc-a610-f76638fda300 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:52.606347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d828c-7824-41e3-9771-c5cc344725cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:52.607055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d828c-7824-41e3-9771-c5cc344725cd admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:52.607343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d828c-7824-41e3-9771-c5cc344725cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:52.607626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d828c-7824-41e3-9771-c5cc344725cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:52.609929 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d828c-7824-41e3-9771-c5cc344725cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:52.610852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d828c-7824-41e3-9771-c5cc344725cd admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:52.687187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782b329-e7e8-4d80-b4fc-0af3bd70656b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:52.690535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1013/2029] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:52 2022] POST /identity/v3/projects => generated 391 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:52.818815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:52.851040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782b329-e7e8-4d80-b4fc-0af3bd70656b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:52.948578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009927f-7284-4f9a-b43e-cd4011e93025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:52.949849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009927f-7284-4f9a-b43e-cd4011e93025 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:52.950212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009927f-7284-4f9a-b43e-cd4011e93025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:52.950593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009927f-7284-4f9a-b43e-cd4011e93025 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:52.951432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009927f-7284-4f9a-b43e-cd4011e93025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:52.953589 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009927f-7284-4f9a-b43e-cd4011e93025 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:52.986629 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1017/2030] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:52 2022] POST /identity/v3/users => generated 302 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:34:52.994044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d828c-7824-41e3-9771-c5cc344725cd admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:53.020370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d828c-7824-41e3-9771-c5cc344725cd admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:53.026558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd94c4-b56b-4b70-bbbc-940cf7f7ba2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:53.027533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd94c4-b56b-4b70-bbbc-940cf7f7ba2c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:53.028354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd94c4-b56b-4b70-bbbc-940cf7f7ba2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:53.029423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd94c4-b56b-4b70-bbbc-940cf7f7ba2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:53.029777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd94c4-b56b-4b70-bbbc-940cf7f7ba2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:53.033971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd94c4-b56b-4b70-bbbc-940cf7f7ba2c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:53.043591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:53.101232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1014/2031] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:53.102112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009927f-7284-4f9a-b43e-cd4011e93025 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:53.118308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009927f-7284-4f9a-b43e-cd4011e93025 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:53.123705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52c186-765e-48b6-b3b9-a7ba8b503332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:53.124649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52c186-765e-48b6-b3b9-a7ba8b503332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:53.124931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52c186-765e-48b6-b3b9-a7ba8b503332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:53.125208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52c186-765e-48b6-b3b9-a7ba8b503332 admin admin] PATH_INFO: `/v3/projects/3c17469c63464d90bce7bb035b575f3d/users/3e4a650efa244798a0dd94b5eb0893fe/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:53.147116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52c186-765e-48b6-b3b9-a7ba8b503332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c17469c63464d90bce7bb035b575f3d, user_id=3e4a650efa244798a0dd94b5eb0893fe, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:53.150365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52c186-765e-48b6-b3b9-a7ba8b503332 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:53.174079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1018/2032] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:53 2022] PUT /identity/v3/projects/3c17469c63464d90bce7bb035b575f3d/users/3e4a650efa244798a0dd94b5eb0893fe/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:53.177863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd94c4-b56b-4b70-bbbc-940cf7f7ba2c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:53.189461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd94c4-b56b-4b70-bbbc-940cf7f7ba2c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:53.267388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785d5d8-c81f-4c90-add9-1817f245b74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:53.268172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785d5d8-c81f-4c90-add9-1817f245b74e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:53.268783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785d5d8-c81f-4c90-add9-1817f245b74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:53.269030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785d5d8-c81f-4c90-add9-1817f245b74e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:53.269734 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785d5d8-c81f-4c90-add9-1817f245b74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:53.271501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785d5d8-c81f-4c90-add9-1817f245b74e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:53.284735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1015/2033] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:53.289117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52c186-765e-48b6-b3b9-a7ba8b503332 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:53.302054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52c186-765e-48b6-b3b9-a7ba8b503332 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:53.308692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a7b09-d11e-4594-a283-a9180a9e0232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:53.309779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a7b09-d11e-4594-a283-a9180a9e0232 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:53.310075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a7b09-d11e-4594-a283-a9180a9e0232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:53.310437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a7b09-d11e-4594-a283-a9180a9e0232 admin admin] PATH_INFO: `/v3/projects/3c17469c63464d90bce7bb035b575f3d/users/3e4a650efa244798a0dd94b5eb0893fe/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:53.315047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a7b09-d11e-4594-a283-a9180a9e0232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c17469c63464d90bce7bb035b575f3d, user_id=3e4a650efa244798a0dd94b5eb0893fe, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:53.317967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a7b09-d11e-4594-a283-a9180a9e0232 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:53.344199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1019/2034] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:53 2022] PUT /identity/v3/projects/3c17469c63464d90bce7bb035b575f3d/users/3e4a650efa244798a0dd94b5eb0893fe/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:53.345040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ab1f7-d7e6-4e7d-a343-fb2ccacf378d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:53.345344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ab1f7-d7e6-4e7d-a343-fb2ccacf378d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:53.346186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ab1f7-d7e6-4e7d-a343-fb2ccacf378d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:53.390714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6e2ab1f7-d7e6-4e7d-a343-fb2ccacf378d None None] MFA Rules not processed for user `3e4a650efa244798a0dd94b5eb0893fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:53.391987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6e2ab1f7-d7e6-4e7d-a343-fb2ccacf378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:53.490967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7517d-65b6-427c-8687-286a048cfa23 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:53.491291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7517d-65b6-427c-8687-286a048cfa23 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:53.491534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7517d-65b6-427c-8687-286a048cfa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:53.530603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1016/2035] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:53.532042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ab1f7-d7e6-4e7d-a343-fb2ccacf378d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:53.533529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6e2ab1f7-d7e6-4e7d-a343-fb2ccacf378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:53.546558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-31a7517d-65b6-427c-8687-286a048cfa23 None None] MFA Rules not processed for user `ac4cfa3ccdab43c3adafcef62e62365f`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:53.547795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-31a7517d-65b6-427c-8687-286a048cfa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:53.669380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ab1f7-d7e6-4e7d-a343-fb2ccacf378d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:53.676126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:53.680772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:53.683542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:53.683542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] PATH_INFO: `/v3/projects/6668f7226a8247c1804cc5b82f8f688c` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:53.695011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=6668f7226a8247c1804cc5b82f8f688c)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:53.700938 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:53.732559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1020/2036] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:53.735578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7517d-65b6-427c-8687-286a048cfa23 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:53.737286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-31a7517d-65b6-427c-8687-286a048cfa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:53.770634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:53.804465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6668f7226a8247c1804cc5b82f8f688c', 'request_id': 'req-532a1102-5c2f-4062-b743-3602430960ce'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:53.810357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:53.814105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6668f7226a8247c1804cc5b82f8f688c', 'request_id': 'req-532a1102-5c2f-4062-b743-3602430960ce'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:53.821836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6668f7226a8247c1804cc5b82f8f688c', 'request_id': 'req-532a1102-5c2f-4062-b743-3602430960ce'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:53.840830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] The token cache is being invalidate because project 6668f7226a8247c1804cc5b82f8f688c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:53.841171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:53.842429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-532a1102-5c2f-4062-b743-3602430960ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:53.843148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7517d-65b6-427c-8687-286a048cfa23 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:53.844280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1017/2037] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:53 2022] DELETE /identity/v3/projects/6668f7226a8247c1804cc5b82f8f688c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:53.849132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b0faa-e3c6-4c20-91fa-8bd20203afd8 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:53.849480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b0faa-e3c6-4c20-91fa-8bd20203afd8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:53.849764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b0faa-e3c6-4c20-91fa-8bd20203afd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:53.879436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-796b0faa-e3c6-4c20-91fa-8bd20203afd8 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:53.883083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-796b0faa-e3c6-4c20-91fa-8bd20203afd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:53.914727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76898a-e1e2-46ae-88c5-64cc51134aa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:53.915234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76898a-e1e2-46ae-88c5-64cc51134aa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:53.915527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76898a-e1e2-46ae-88c5-64cc51134aa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:53.915876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76898a-e1e2-46ae-88c5-64cc51134aa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:53.918189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0e76898a-e1e2-46ae-88c5-64cc51134aa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:53.976974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1018/2038] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:53.986381 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e9cda-61f9-4142-b2eb-aebb278005f0 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:53.986974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e9cda-61f9-4142-b2eb-aebb278005f0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:53.987465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e9cda-61f9-4142-b2eb-aebb278005f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.013505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8f4e9cda-61f9-4142-b2eb-aebb278005f0 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:54.015030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8f4e9cda-61f9-4142-b2eb-aebb278005f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:54.022176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76898a-e1e2-46ae-88c5-64cc51134aa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:54.024132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76898a-e1e2-46ae-88c5-64cc51134aa7 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:54.027545 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1019/2039] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:54.032373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e9cda-61f9-4142-b2eb-aebb278005f0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:54.039429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1021/2040] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:54.044252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e9cda-61f9-4142-b2eb-aebb278005f0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:54.049918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1305eee-e753-4786-8450-91117ee29b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:54.050666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1305eee-e753-4786-8450-91117ee29b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:54.051015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1305eee-e753-4786-8450-91117ee29b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:54.051359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1305eee-e753-4786-8450-91117ee29b7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.052592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1305eee-e753-4786-8450-91117ee29b7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:34:54.052987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1305eee-e753-4786-8450-91117ee29b7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:54.055048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1305eee-e753-4786-8450-91117ee29b7f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:54.069283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1020/2041] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:34:54.073549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76898a-e1e2-46ae-88c5-64cc51134aa7 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:54.085692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76898a-e1e2-46ae-88c5-64cc51134aa7 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:54.091835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c12817-1fa7-4515-a92b-5ffc456f5187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:54.092568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c12817-1fa7-4515-a92b-5ffc456f5187 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:54.096380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c12817-1fa7-4515-a92b-5ffc456f5187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:54.096380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c12817-1fa7-4515-a92b-5ffc456f5187 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.096380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c12817-1fa7-4515-a92b-5ffc456f5187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:54.096380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c12817-1fa7-4515-a92b-5ffc456f5187 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:54.102954 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1305eee-e753-4786-8450-91117ee29b7f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:54.106127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d1305eee-e753-4786-8450-91117ee29b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:54.123942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1305eee-e753-4786-8450-91117ee29b7f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:54.129740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ca264-ff9e-4f57-a283-663ad5d61783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:54.131297 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ca264-ff9e-4f57-a283-663ad5d61783 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:54.131814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ca264-ff9e-4f57-a283-663ad5d61783 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:54.132237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ca264-ff9e-4f57-a283-663ad5d61783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.135573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f14ca264-ff9e-4f57-a283-663ad5d61783 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:54.136192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1022/2042] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:34:54 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:54.137122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c12817-1fa7-4515-a92b-5ffc456f5187 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:54.229731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c12817-1fa7-4515-a92b-5ffc456f5187 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:54.233724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ca264-ff9e-4f57-a283-663ad5d61783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:54.234648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8fdd3-9362-463b-ad6a-d7806a9819b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:54.235318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8fdd3-9362-463b-ad6a-d7806a9819b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:54.235600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8fdd3-9362-463b-ad6a-d7806a9819b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:54.235878 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8fdd3-9362-463b-ad6a-d7806a9819b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.236062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ca264-ff9e-4f57-a283-663ad5d61783 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:54.236598 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8fdd3-9362-463b-ad6a-d7806a9819b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:54.238678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8fdd3-9362-463b-ad6a-d7806a9819b5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:54.248495 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1021/2043] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:54.272749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1023/2044] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:34:54 2022] POST /identity/v3/users => generated 286 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:34:54.278510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ca264-ff9e-4f57-a283-663ad5d61783 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:54.289765 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ca264-ff9e-4f57-a283-663ad5d61783 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:54.296035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f05bc-6054-41b6-99a6-c0a4ef1a5d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:54.296035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f05bc-6054-41b6-99a6-c0a4ef1a5d69 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:54.296035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f05bc-6054-41b6-99a6-c0a4ef1a5d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:54.296035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f05bc-6054-41b6-99a6-c0a4ef1a5d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.297047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f05bc-6054-41b6-99a6-c0a4ef1a5d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:54.298742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f05bc-6054-41b6-99a6-c0a4ef1a5d69 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:54.313312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1022/2045] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:54.314508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8fdd3-9362-463b-ad6a-d7806a9819b5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:54.326562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8fdd3-9362-463b-ad6a-d7806a9819b5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:54.331945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4a519a-0032-4b01-9804-41e333b26a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:54.332678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a519a-0032-4b01-9804-41e333b26a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:54.332952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a519a-0032-4b01-9804-41e333b26a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:54.333230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a519a-0032-4b01-9804-41e333b26a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.334050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4a519a-0032-4b01-9804-41e333b26a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:54.335848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4a519a-0032-4b01-9804-41e333b26a4b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:54.342969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:54.353865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f05bc-6054-41b6-99a6-c0a4ef1a5d69 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:54.360549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1024/2046] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:34:54.367106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f05bc-6054-41b6-99a6-c0a4ef1a5d69 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:54.387674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e5ec0-a09a-4305-8a94-4932534fd02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:54.388738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e5ec0-a09a-4305-8a94-4932534fd02c admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:54.389090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e5ec0-a09a-4305-8a94-4932534fd02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:54.389737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e5ec0-a09a-4305-8a94-4932534fd02c admin admin] PATH_INFO: `/v3/projects/2443ea7c59e14e6e8eba6d63f4b7936e/users/6fd87993dc934c80aff8b1563227e4a0/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.417017 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e5ec0-a09a-4305-8a94-4932534fd02c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2443ea7c59e14e6e8eba6d63f4b7936e, user_id=6fd87993dc934c80aff8b1563227e4a0, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:54.421215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e5ec0-a09a-4305-8a94-4932534fd02c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:54.423980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:54.440290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1023/2047] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:34:54 2022] PUT /identity/v3/projects/2443ea7c59e14e6e8eba6d63f4b7936e/users/6fd87993dc934c80aff8b1563227e4a0/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:34:54.451395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f48ee-60e8-4bf6-b791-59df03347124 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:54.451652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f48ee-60e8-4bf6-b791-59df03347124 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:54.452330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f48ee-60e8-4bf6-b791-59df03347124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.486468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d84f48ee-60e8-4bf6-b791-59df03347124 None None] MFA Rules not processed for user `6fd87993dc934c80aff8b1563227e4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:54.488201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d84f48ee-60e8-4bf6-b791-59df03347124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:54.563472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e5ec0-a09a-4305-8a94-4932534fd02c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:54.565693 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7d3e5ec0-a09a-4305-8a94-4932534fd02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:54.594217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1025/2048] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:34:54.653554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e5ec0-a09a-4305-8a94-4932534fd02c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:54.671448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986b078-25c5-4539-a800-3e6c4ed4f6d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:54.671448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986b078-25c5-4539-a800-3e6c4ed4f6d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:54.671448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986b078-25c5-4539-a800-3e6c4ed4f6d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:54.671448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986b078-25c5-4539-a800-3e6c4ed4f6d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.671714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c986b078-25c5-4539-a800-3e6c4ed4f6d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:54.682470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f48ee-60e8-4bf6-b791-59df03347124 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:54.696988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f48ee-60e8-4bf6-b791-59df03347124 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:54.706554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f63c12-a075-46ef-8ca6-efcdb9f0848d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:54.706554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f63c12-a075-46ef-8ca6-efcdb9f0848d service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:54.706869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f63c12-a075-46ef-8ca6-efcdb9f0848d service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:54.707120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f63c12-a075-46ef-8ca6-efcdb9f0848d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:54.719444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f63c12-a075-46ef-8ca6-efcdb9f0848d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:54.721650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f63c12-a075-46ef-8ca6-efcdb9f0848d service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:54.741529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1026/2049] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:34:54.767783 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986b078-25c5-4539-a800-3e6c4ed4f6d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:54.769376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986b078-25c5-4539-a800-3e6c4ed4f6d3 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:54.780165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1024/2050] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:55.940039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf74c8-8623-4252-befd-869df7b8bd55 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:55.940535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf74c8-8623-4252-befd-869df7b8bd55 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:55.940625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf74c8-8623-4252-befd-869df7b8bd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:55.973544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c9bf74c8-8623-4252-befd-869df7b8bd55 None None] MFA Rules not processed for user `ac4cfa3ccdab43c3adafcef62e62365f`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:55.974893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c9bf74c8-8623-4252-befd-869df7b8bd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:56.041875 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1027/2051] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:58.483664 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986b078-25c5-4539-a800-3e6c4ed4f6d3 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:58.485369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c986b078-25c5-4539-a800-3e6c4ed4f6d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:58.560810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986b078-25c5-4539-a800-3e6c4ed4f6d3 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:58.566155 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:58.567125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:58.567406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:58.567683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] PATH_INFO: `/v3/users/358425a7d080425185241a7690faa0ec` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:58.568972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=358425a7d080425185241a7690faa0ec)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:58.570901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:58.627809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '358425a7d080425185241a7690faa0ec'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:58.628581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '358425a7d080425185241a7690faa0ec'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:58.629246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '358425a7d080425185241a7690faa0ec'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:58.633855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:INSERT=1,keystone:SELECT=92,keystone:DELETE=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:58.644405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '358425a7d080425185241a7690faa0ec'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:58.662603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '358425a7d080425185241a7690faa0ec'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:58.705953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1025/2052] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:58 2022] DELETE /identity/v3/users/358425a7d080425185241a7690faa0ec => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:58.721813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cda62d-f3ba-4a00-b60d-9166a076d7c5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:58.722320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cda62d-f3ba-4a00-b60d-9166a076d7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:58.722404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cda62d-f3ba-4a00-b60d-9166a076d7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:58.749736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-25cda62d-f3ba-4a00-b60d-9166a076d7c5 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:34:58.751801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-25cda62d-f3ba-4a00-b60d-9166a076d7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:58.829457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1028/2053] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:34:58.849060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:58.850857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:58.928060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794946e-9845-4bbb-be16-96cc25e9f7f8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:58.936762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fffab-31a8-4ada-9220-d5a2e3270266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:58.937673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fffab-31a8-4ada-9220-d5a2e3270266 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:58.937960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fffab-31a8-4ada-9220-d5a2e3270266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:58.938237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fffab-31a8-4ada-9220-d5a2e3270266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:58.951787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fffab-31a8-4ada-9220-d5a2e3270266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:58.953876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fffab-31a8-4ada-9220-d5a2e3270266 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:58.963767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1026/2054] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:34:59.592549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cda62d-f3ba-4a00-b60d-9166a076d7c5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:59.594364 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-25cda62d-f3ba-4a00-b60d-9166a076d7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:34:59.644863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cda62d-f3ba-4a00-b60d-9166a076d7c5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:59.658764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:59.661689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:59.661984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:59.662263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] PATH_INFO: `/v3/projects/3ab92b909dc04e95938bbcf591700fcd` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:59.664262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab92b909dc04e95938bbcf591700fcd)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:59.668056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:59.693904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:34:59.750096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab92b909dc04e95938bbcf591700fcd', 'request_id': 'req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:59.758350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab92b909dc04e95938bbcf591700fcd', 'request_id': 'req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:59.771016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab92b909dc04e95938bbcf591700fcd', 'request_id': 'req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:59.785091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] The token cache is being invalidate because project 3ab92b909dc04e95938bbcf591700fcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:34:59.785391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:59.786158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:59.787893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1029/2055] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:34:59 2022] DELETE /identity/v3/projects/3ab92b909dc04e95938bbcf591700fcd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:34:59.792748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fffab-31a8-4ada-9220-d5a2e3270266 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:34:59.803837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fffab-31a8-4ada-9220-d5a2e3270266 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:34:59.909945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:34:59.911066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:34:59.911413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:34:59.911709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] PATH_INFO: `/v3/users/3e4a650efa244798a0dd94b5eb0893fe` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:34:59.918755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e4a650efa244798a0dd94b5eb0893fe)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:34:59.918755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:34:59.985932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e4a650efa244798a0dd94b5eb0893fe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:59.986984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e4a650efa244798a0dd94b5eb0893fe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:59.988077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e4a650efa244798a0dd94b5eb0893fe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:34:59.998971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e4a650efa244798a0dd94b5eb0893fe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:00.012976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:00.014379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-efd5b3a1-d36a-4fb5-8b93-79f66050572b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e4a650efa244798a0dd94b5eb0893fe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:00.039898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1027/2056] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:34:59 2022] DELETE /identity/v3/users/3e4a650efa244798a0dd94b5eb0893fe => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:00.597258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:00.598751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:00.626405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f5d1-ca36-42e3-9c7f-92774cb1e85c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:00.691876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:00.692804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:00.693096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:00.693401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] PATH_INFO: `/v3/projects/3c17469c63464d90bce7bb035b575f3d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:00.694681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c17469c63464d90bce7bb035b575f3d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:00.696681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:00.762847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c17469c63464d90bce7bb035b575f3d', 'request_id': 'req-4d0e4b3e-4faa-46da-8754-be74ec4898e9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:00.771970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c17469c63464d90bce7bb035b575f3d', 'request_id': 'req-4d0e4b3e-4faa-46da-8754-be74ec4898e9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:00.784940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c17469c63464d90bce7bb035b575f3d', 'request_id': 'req-4d0e4b3e-4faa-46da-8754-be74ec4898e9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:00.802665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] The token cache is being invalidate because project 3c17469c63464d90bce7bb035b575f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:00.802957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:00.803821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d0e4b3e-4faa-46da-8754-be74ec4898e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:00.805899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1030/2057] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:00 2022] DELETE /identity/v3/projects/3c17469c63464d90bce7bb035b575f3d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:00.810958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7151622-201a-4e20-924c-5acd2dd35926 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:00.811533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7151622-201a-4e20-924c-5acd2dd35926 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:00.811533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7151622-201a-4e20-924c-5acd2dd35926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:00.835196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a7151622-201a-4e20-924c-5acd2dd35926 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:00.836553 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a7151622-201a-4e20-924c-5acd2dd35926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:00.924704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1028/2058] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:35:00.933534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ff8c6-533a-42b9-b1a0-edfe41373033 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:00.933684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ff8c6-533a-42b9-b1a0-edfe41373033 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:00.933862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ff8c6-533a-42b9-b1a0-edfe41373033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:00.959958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b36ff8c6-533a-42b9-b1a0-edfe41373033 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:00.961342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b36ff8c6-533a-42b9-b1a0-edfe41373033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:00.997008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7151622-201a-4e20-924c-5acd2dd35926 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:01.000710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1031/2059] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:01.007551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7151622-201a-4e20-924c-5acd2dd35926 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:01.013494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb418241-8bce-4123-ae53-db5f6267ece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:01.014349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb418241-8bce-4123-ae53-db5f6267ece3 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.014476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb418241-8bce-4123-ae53-db5f6267ece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.014692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb418241-8bce-4123-ae53-db5f6267ece3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.015421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb418241-8bce-4123-ae53-db5f6267ece3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:35:01.015727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb418241-8bce-4123-ae53-db5f6267ece3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:01.017400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb418241-8bce-4123-ae53-db5f6267ece3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:01.030380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1029/2060] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:35:01.034856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ff8c6-533a-42b9-b1a0-edfe41373033 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:01.047295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ff8c6-533a-42b9-b1a0-edfe41373033 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:01.052283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8463ad-fab9-4d6a-8743-e560b3b56ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:01.052964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8463ad-fab9-4d6a-8743-e560b3b56ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.053253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8463ad-fab9-4d6a-8743-e560b3b56ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.053532 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8463ad-fab9-4d6a-8743-e560b3b56ba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.054229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8463ad-fab9-4d6a-8743-e560b3b56ba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:01.056714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8463ad-fab9-4d6a-8743-e560b3b56ba5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:01.085295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1032/2061] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:01 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:35:01.090686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb418241-8bce-4123-ae53-db5f6267ece3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:01.110673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb418241-8bce-4123-ae53-db5f6267ece3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:01.188900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe984a4-4660-41e8-ae9e-5bba69ec062a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:01.189726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe984a4-4660-41e8-ae9e-5bba69ec062a admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.190124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe984a4-4660-41e8-ae9e-5bba69ec062a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.190538 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe984a4-4660-41e8-ae9e-5bba69ec062a admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.191364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe984a4-4660-41e8-ae9e-5bba69ec062a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:01.193638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe984a4-4660-41e8-ae9e-5bba69ec062a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:01.220139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1030/2062] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:01 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:01.224283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8463ad-fab9-4d6a-8743-e560b3b56ba5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:01.236954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8463ad-fab9-4d6a-8743-e560b3b56ba5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:01.242220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8167b0b-5f40-478c-a8cd-f7e85902e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:01.243045 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8167b0b-5f40-478c-a8cd-f7e85902e46d admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.243421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8167b0b-5f40-478c-a8cd-f7e85902e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.243835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8167b0b-5f40-478c-a8cd-f7e85902e46d admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.244742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8167b0b-5f40-478c-a8cd-f7e85902e46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:01.247450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8167b0b-5f40-478c-a8cd-f7e85902e46d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:01.249327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:01.264744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1033/2063] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:01.268132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe984a4-4660-41e8-ae9e-5bba69ec062a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:01.280543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe984a4-4660-41e8-ae9e-5bba69ec062a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:01.285291 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcf44c-6541-4fb9-9c1f-bf8e10af7e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:01.286037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf44c-6541-4fb9-9c1f-bf8e10af7e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.286391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf44c-6541-4fb9-9c1f-bf8e10af7e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.286739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf44c-6541-4fb9-9c1f-bf8e10af7e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.287539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcf44c-6541-4fb9-9c1f-bf8e10af7e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:01.289281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcf44c-6541-4fb9-9c1f-bf8e10af7e68 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:01.301625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1031/2064] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:01.305847 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8167b0b-5f40-478c-a8cd-f7e85902e46d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:01.317081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8167b0b-5f40-478c-a8cd-f7e85902e46d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:01.322470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6982a1-069d-4948-9de5-10cc113f3aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:01.323925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6982a1-069d-4948-9de5-10cc113f3aa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.323925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6982a1-069d-4948-9de5-10cc113f3aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.324157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6982a1-069d-4948-9de5-10cc113f3aa6 admin admin] PATH_INFO: `/v3/projects/d124593615b44eb7a56b05a2d1503eca/users/8000f977a1de4e16a0090bce7b33fb5c/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.341028 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6982a1-069d-4948-9de5-10cc113f3aa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d124593615b44eb7a56b05a2d1503eca, user_id=8000f977a1de4e16a0090bce7b33fb5c, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:01.343832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6982a1-069d-4948-9de5-10cc113f3aa6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:01.358913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1034/2065] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:01 2022] PUT /identity/v3/projects/d124593615b44eb7a56b05a2d1503eca/users/8000f977a1de4e16a0090bce7b33fb5c/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:01.369460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eafe4-bbca-460b-9153-44fe0797d594 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.369758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eafe4-bbca-460b-9153-44fe0797d594 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.370038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eafe4-bbca-460b-9153-44fe0797d594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.404084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cc7eafe4-bbca-460b-9153-44fe0797d594 None None] MFA Rules not processed for user `8000f977a1de4e16a0090bce7b33fb5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:01.405334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cc7eafe4-bbca-460b-9153-44fe0797d594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:01.454678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:01.480010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1032/2066] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:01.491039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6982a1-069d-4948-9de5-10cc113f3aa6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:01.492842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bd6982a1-069d-4948-9de5-10cc113f3aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:01.554873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6982a1-069d-4948-9de5-10cc113f3aa6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:01.559022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b0054-d14a-4360-947d-c30edf2c11ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:01.559631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b0054-d14a-4360-947d-c30edf2c11ae service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.559911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b0054-d14a-4360-947d-c30edf2c11ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.560193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b0054-d14a-4360-947d-c30edf2c11ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.568849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b0054-d14a-4360-947d-c30edf2c11ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:01.570274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b0054-d14a-4360-947d-c30edf2c11ae service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:01.580159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1035/2067] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:01.601546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eafe4-bbca-460b-9153-44fe0797d594 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:01.674266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eafe4-bbca-460b-9153-44fe0797d594 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:01.678912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:01.679799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.680083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.680397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] PATH_INFO: `/v3/users/8000f977a1de4e16a0090bce7b33fb5c` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.682562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8000f977a1de4e16a0090bce7b33fb5c)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:01.684598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:01.745439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8000f977a1de4e16a0090bce7b33fb5c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:01.746230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8000f977a1de4e16a0090bce7b33fb5c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:01.747303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8000f977a1de4e16a0090bce7b33fb5c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:01.760334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8000f977a1de4e16a0090bce7b33fb5c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:01.777811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8000f977a1de4e16a0090bce7b33fb5c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:01.794106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1033/2068] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:01 2022] DELETE /identity/v3/users/8000f977a1de4e16a0090bce7b33fb5c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:01.798329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fcfb3-5a77-4bda-854d-6d6882f7cb92 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.798885 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fcfb3-5a77-4bda-854d-6d6882f7cb92 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.798996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fcfb3-5a77-4bda-854d-6d6882f7cb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.820070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-708fcfb3-5a77-4bda-854d-6d6882f7cb92 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:01.821413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-708fcfb3-5a77-4bda-854d-6d6882f7cb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:01.887275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1036/2069] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:01.897315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:01.898796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:01.969886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1b3dc-1bba-4142-ae16-1b587012540d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:01.974151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71452671-8a7a-4914-8fea-3193a89e1962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:01.975083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71452671-8a7a-4914-8fea-3193a89e1962 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:01.975529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71452671-8a7a-4914-8fea-3193a89e1962 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:01.975965 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71452671-8a7a-4914-8fea-3193a89e1962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:01.990091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71452671-8a7a-4914-8fea-3193a89e1962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:01.991797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71452671-8a7a-4914-8fea-3193a89e1962 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:02.004146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1034/2070] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:02.342860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fcfb3-5a77-4bda-854d-6d6882f7cb92 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:02.344437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-708fcfb3-5a77-4bda-854d-6d6882f7cb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:02.365154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fcfb3-5a77-4bda-854d-6d6882f7cb92 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:02.372401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:02.373423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:02.373832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:02.374737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] PATH_INFO: `/v3/projects/d124593615b44eb7a56b05a2d1503eca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:02.375960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d124593615b44eb7a56b05a2d1503eca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:02.378053 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:02.426277 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd124593615b44eb7a56b05a2d1503eca', 'request_id': 'req-19863e21-f863-432b-8b44-6a4e8cae2ad2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:02.433016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd124593615b44eb7a56b05a2d1503eca', 'request_id': 'req-19863e21-f863-432b-8b44-6a4e8cae2ad2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:02.440527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd124593615b44eb7a56b05a2d1503eca', 'request_id': 'req-19863e21-f863-432b-8b44-6a4e8cae2ad2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:02.448202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:02.453853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] The token cache is being invalidate because project d124593615b44eb7a56b05a2d1503eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:02.454390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:02.455473 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-19863e21-f863-432b-8b44-6a4e8cae2ad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:02.457843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1037/2071] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:02 2022] DELETE /identity/v3/projects/d124593615b44eb7a56b05a2d1503eca => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:02.463606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a505a9-3f1b-48ba-bf95-4a0332240950 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:02.463892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a505a9-3f1b-48ba-bf95-4a0332240950 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:02.464171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a505a9-3f1b-48ba-bf95-4a0332240950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:02.476554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:02.484287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-76a505a9-3f1b-48ba-bf95-4a0332240950 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:02.485650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-76a505a9-3f1b-48ba-bf95-4a0332240950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:02.544582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1035/2072] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:02.553266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb620cf5-b954-4116-8799-60d74ef62579 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:02.553552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb620cf5-b954-4116-8799-60d74ef62579 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:02.553833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb620cf5-b954-4116-8799-60d74ef62579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:02.572040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-cb620cf5-b954-4116-8799-60d74ef62579 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:02.573538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cb620cf5-b954-4116-8799-60d74ef62579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:02.581488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1038/2073] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:02.586080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a505a9-3f1b-48ba-bf95-4a0332240950 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:02.596248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a505a9-3f1b-48ba-bf95-4a0332240950 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:02.600896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356543f7-93b4-44d3-8c65-8fa771f4e5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:02.601610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356543f7-93b4-44d3-8c65-8fa771f4e5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:02.601886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356543f7-93b4-44d3-8c65-8fa771f4e5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:02.602157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356543f7-93b4-44d3-8c65-8fa771f4e5f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:02.603515 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356543f7-93b4-44d3-8c65-8fa771f4e5f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:35:02.603930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356543f7-93b4-44d3-8c65-8fa771f4e5f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:02.605632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356543f7-93b4-44d3-8c65-8fa771f4e5f4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:02.618962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1036/2074] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:35:02.623403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb620cf5-b954-4116-8799-60d74ef62579 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:02.632782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb620cf5-b954-4116-8799-60d74ef62579 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:02.637842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7662c4c-c221-45e3-9d60-8bec057f109b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:02.638498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7662c4c-c221-45e3-9d60-8bec057f109b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:02.638788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7662c4c-c221-45e3-9d60-8bec057f109b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:02.639071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7662c4c-c221-45e3-9d60-8bec057f109b admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:02.639763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7662c4c-c221-45e3-9d60-8bec057f109b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:02.641714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7662c4c-c221-45e3-9d60-8bec057f109b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:02.668888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1039/2075] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:02 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:02.673217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356543f7-93b4-44d3-8c65-8fa771f4e5f4 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:02.685079 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356543f7-93b4-44d3-8c65-8fa771f4e5f4 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:02.745608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95370587-be76-4f21-b809-b65136a00f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:02.746609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95370587-be76-4f21-b809-b65136a00f2a admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:02.746896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95370587-be76-4f21-b809-b65136a00f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:02.747175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95370587-be76-4f21-b809-b65136a00f2a admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:02.747875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95370587-be76-4f21-b809-b65136a00f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:02.749827 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95370587-be76-4f21-b809-b65136a00f2a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:02.772734 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1037/2076] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:02 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:02.777626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7662c4c-c221-45e3-9d60-8bec057f109b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:02.788736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7662c4c-c221-45e3-9d60-8bec057f109b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:02.794853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a97b0-085a-45ea-a12a-c0000d24543e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:02.795740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a97b0-085a-45ea-a12a-c0000d24543e admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:02.795892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a97b0-085a-45ea-a12a-c0000d24543e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:02.796147 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a97b0-085a-45ea-a12a-c0000d24543e admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:02.797068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a97b0-085a-45ea-a12a-c0000d24543e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:02.798776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a97b0-085a-45ea-a12a-c0000d24543e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:02.810421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1040/2077] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:02.814399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95370587-be76-4f21-b809-b65136a00f2a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:02.826230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95370587-be76-4f21-b809-b65136a00f2a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:02.831617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d1cc6-8985-4817-9b83-901c96b4c6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:02.832212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d1cc6-8985-4817-9b83-901c96b4c6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:02.832526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d1cc6-8985-4817-9b83-901c96b4c6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:02.832811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d1cc6-8985-4817-9b83-901c96b4c6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:02.833547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d1cc6-8985-4817-9b83-901c96b4c6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:02.835353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d1cc6-8985-4817-9b83-901c96b4c6b9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:02.850366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1038/2078] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:02.856918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a97b0-085a-45ea-a12a-c0000d24543e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:02.883399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a97b0-085a-45ea-a12a-c0000d24543e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:02.891793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882aef4-1321-4e0e-beb4-43c5a2ca3ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:02.892936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882aef4-1321-4e0e-beb4-43c5a2ca3ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:02.893279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882aef4-1321-4e0e-beb4-43c5a2ca3ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:02.893598 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882aef4-1321-4e0e-beb4-43c5a2ca3ca1 admin admin] PATH_INFO: `/v3/projects/1001b3d3f617444db7683aba3bd057b9/users/dec1bb559baa4778899056db008f5003/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:02.921276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882aef4-1321-4e0e-beb4-43c5a2ca3ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1001b3d3f617444db7683aba3bd057b9, user_id=dec1bb559baa4778899056db008f5003, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:02.925258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882aef4-1321-4e0e-beb4-43c5a2ca3ca1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:02.944721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1041/2079] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:02 2022] PUT /identity/v3/projects/1001b3d3f617444db7683aba3bd057b9/users/dec1bb559baa4778899056db008f5003/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:02.956942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ba0cd-36c6-43ef-a014-e028639febea None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:02.957507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ba0cd-36c6-43ef-a014-e028639febea None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:02.957507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ba0cd-36c6-43ef-a014-e028639febea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:02.989757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d63ba0cd-36c6-43ef-a014-e028639febea None None] MFA Rules not processed for user `dec1bb559baa4778899056db008f5003`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:02.990748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d63ba0cd-36c6-43ef-a014-e028639febea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:03.083512 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1039/2080] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:03.115798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882aef4-1321-4e0e-beb4-43c5a2ca3ca1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:03.117833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3882aef4-1321-4e0e-beb4-43c5a2ca3ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:03.174481 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882aef4-1321-4e0e-beb4-43c5a2ca3ca1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:03.178107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f4769-0464-4064-862a-282c2988c744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:03.178745 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f4769-0464-4064-862a-282c2988c744 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:03.179027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f4769-0464-4064-862a-282c2988c744 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:03.179300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f4769-0464-4064-862a-282c2988c744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:03.188182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f4769-0464-4064-862a-282c2988c744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:03.189673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f4769-0464-4064-862a-282c2988c744 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:03.199827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1042/2081] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:03.214721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ba0cd-36c6-43ef-a014-e028639febea None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:03.219077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:03.298835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ba0cd-36c6-43ef-a014-e028639febea None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:03.304254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:03.305172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:03.305457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:03.305735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] PATH_INFO: `/v3/users/dec1bb559baa4778899056db008f5003` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:03.308129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dec1bb559baa4778899056db008f5003)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:03.309589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:03.373760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dec1bb559baa4778899056db008f5003'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:03.374453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dec1bb559baa4778899056db008f5003'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:03.375113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dec1bb559baa4778899056db008f5003'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:03.382671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dec1bb559baa4778899056db008f5003'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:03.396274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dec1bb559baa4778899056db008f5003'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:03.413719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1040/2082] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:03 2022] DELETE /identity/v3/users/dec1bb559baa4778899056db008f5003 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:03.420940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36d548-d6ff-44ea-b945-105afe6c1c16 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:03.421383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36d548-d6ff-44ea-b945-105afe6c1c16 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:03.421729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36d548-d6ff-44ea-b945-105afe6c1c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:03.439023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-df36d548-d6ff-44ea-b945-105afe6c1c16 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:03.440270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-df36d548-d6ff-44ea-b945-105afe6c1c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:03.509617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1043/2083] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:03.519351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:03.520916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:03.584443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d257d-c4b1-401c-ba5c-ea691d49f622 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:03.589288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f3d06-26e7-47ab-9e0b-a254c628e1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:03.589879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f3d06-26e7-47ab-9e0b-a254c628e1da service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:03.590163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f3d06-26e7-47ab-9e0b-a254c628e1da service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:03.590442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f3d06-26e7-47ab-9e0b-a254c628e1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:03.600034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f3d06-26e7-47ab-9e0b-a254c628e1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:03.601507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f3d06-26e7-47ab-9e0b-a254c628e1da service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:03.614792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1041/2084] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:03.970450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36d548-d6ff-44ea-b945-105afe6c1c16 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:03.972160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-df36d548-d6ff-44ea-b945-105afe6c1c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:03.989622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36d548-d6ff-44ea-b945-105afe6c1c16 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:03.995069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:03.996021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:03.996844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:03.997246 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] PATH_INFO: `/v3/projects/1001b3d3f617444db7683aba3bd057b9` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:03.998595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1001b3d3f617444db7683aba3bd057b9)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:04.001065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:04.038254 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:04.052250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1001b3d3f617444db7683aba3bd057b9', 'request_id': 'req-d88b926e-8137-48c0-b812-a5b799504a6f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:04.059753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1001b3d3f617444db7683aba3bd057b9', 'request_id': 'req-d88b926e-8137-48c0-b812-a5b799504a6f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:04.068789 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1001b3d3f617444db7683aba3bd057b9', 'request_id': 'req-d88b926e-8137-48c0-b812-a5b799504a6f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:04.083268 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] The token cache is being invalidate because project 1001b3d3f617444db7683aba3bd057b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:04.083522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:04.084169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d88b926e-8137-48c0-b812-a5b799504a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:04.085768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1044/2085] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:03 2022] DELETE /identity/v3/projects/1001b3d3f617444db7683aba3bd057b9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:04.093052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4076a50-132d-4d67-9712-cf16eb756579 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:04.094237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4076a50-132d-4d67-9712-cf16eb756579 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:04.094745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4076a50-132d-4d67-9712-cf16eb756579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:04.117210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f4076a50-132d-4d67-9712-cf16eb756579 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:04.119065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f4076a50-132d-4d67-9712-cf16eb756579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:04.144723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:04.191828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1042/2086] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:04.201097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef022a3-8810-435f-9fa6-7cf6da0cd7e7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:04.201447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef022a3-8810-435f-9fa6-7cf6da0cd7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:04.201691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef022a3-8810-435f-9fa6-7cf6da0cd7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:04.230488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-eef022a3-8810-435f-9fa6-7cf6da0cd7e7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:04.231748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-eef022a3-8810-435f-9fa6-7cf6da0cd7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:04.244415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1045/2087] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:04.248821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4076a50-132d-4d67-9712-cf16eb756579 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:04.259475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4076a50-132d-4d67-9712-cf16eb756579 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:04.265731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a95d0-b974-43d7-a42e-a145bb2dfdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:04.266407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a95d0-b974-43d7-a42e-a145bb2dfdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:04.266692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a95d0-b974-43d7-a42e-a145bb2dfdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:04.266971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a95d0-b974-43d7-a42e-a145bb2dfdf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:04.267706 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a95d0-b974-43d7-a42e-a145bb2dfdf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:35:04.268013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a95d0-b974-43d7-a42e-a145bb2dfdf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:04.269651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a95d0-b974-43d7-a42e-a145bb2dfdf5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:04.284382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1043/2088] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:35:04.288749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef022a3-8810-435f-9fa6-7cf6da0cd7e7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:04.298826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef022a3-8810-435f-9fa6-7cf6da0cd7e7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:04.303501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36204f-82f2-481a-9e2c-6e8f09a059ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:04.304149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36204f-82f2-481a-9e2c-6e8f09a059ad admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:04.304468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36204f-82f2-481a-9e2c-6e8f09a059ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:04.304747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36204f-82f2-481a-9e2c-6e8f09a059ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:04.305477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36204f-82f2-481a-9e2c-6e8f09a059ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:04.308030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36204f-82f2-481a-9e2c-6e8f09a059ad admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:04.333457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1046/2089] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:04 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:04.337713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a95d0-b974-43d7-a42e-a145bb2dfdf5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:04.347842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a95d0-b974-43d7-a42e-a145bb2dfdf5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:04.405435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8157a4-d444-4ce7-b23c-167ddf962e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:04.406019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8157a4-d444-4ce7-b23c-167ddf962e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:04.406308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8157a4-d444-4ce7-b23c-167ddf962e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:04.406580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8157a4-d444-4ce7-b23c-167ddf962e55 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:04.407274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8157a4-d444-4ce7-b23c-167ddf962e55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:04.409246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8157a4-d444-4ce7-b23c-167ddf962e55 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:04.439491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1044/2090] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:04 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:04.444122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36204f-82f2-481a-9e2c-6e8f09a059ad admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:04.456335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36204f-82f2-481a-9e2c-6e8f09a059ad admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:04.461742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f58e-09f3-4827-b9f0-77884c5a3d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:04.462655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f58e-09f3-4827-b9f0-77884c5a3d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:04.463119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f58e-09f3-4827-b9f0-77884c5a3d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:04.463475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f58e-09f3-4827-b9f0-77884c5a3d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:04.464381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028f58e-09f3-4827-b9f0-77884c5a3d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:04.466234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028f58e-09f3-4827-b9f0-77884c5a3d35 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:04.485488 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8157a4-d444-4ce7-b23c-167ddf962e55 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:04.492196 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1047/2091] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:35:04.497247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8157a4-d444-4ce7-b23c-167ddf962e55 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:04.502734 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee90cf-602d-46c3-b9a9-040ca00e8f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:04.503533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee90cf-602d-46c3-b9a9-040ca00e8f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:04.503924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee90cf-602d-46c3-b9a9-040ca00e8f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:04.504369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee90cf-602d-46c3-b9a9-040ca00e8f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:04.505229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee90cf-602d-46c3-b9a9-040ca00e8f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:04.507191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee90cf-602d-46c3-b9a9-040ca00e8f27 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:04.522652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1045/2092] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:04.523928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f58e-09f3-4827-b9f0-77884c5a3d35 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:04.536010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f58e-09f3-4827-b9f0-77884c5a3d35 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:04.541646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3364f6-273c-4093-b3b0-987f4bacd03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:04.543195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3364f6-273c-4093-b3b0-987f4bacd03b admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:04.543195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3364f6-273c-4093-b3b0-987f4bacd03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:04.543558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3364f6-273c-4093-b3b0-987f4bacd03b admin admin] PATH_INFO: `/v3/projects/a50941a4e92849c6a89bcfd6558f5a27/users/7dce02643e89494db02134957ba8407e/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:04.570015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3364f6-273c-4093-b3b0-987f4bacd03b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50941a4e92849c6a89bcfd6558f5a27, user_id=7dce02643e89494db02134957ba8407e, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:04.573980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3364f6-273c-4093-b3b0-987f4bacd03b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:04.586776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1048/2093] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:04 2022] PUT /identity/v3/projects/a50941a4e92849c6a89bcfd6558f5a27/users/7dce02643e89494db02134957ba8407e/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:35:04.597816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cfe8e-addf-4d4c-ab86-b975cf2ae560 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:04.598110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cfe8e-addf-4d4c-ab86-b975cf2ae560 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:04.598394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cfe8e-addf-4d4c-ab86-b975cf2ae560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:04.625630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-081cfe8e-addf-4d4c-ab86-b975cf2ae560 None None] MFA Rules not processed for user `7dce02643e89494db02134957ba8407e`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:04.626863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-081cfe8e-addf-4d4c-ab86-b975cf2ae560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:04.694768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1046/2094] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:04.706110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3364f6-273c-4093-b3b0-987f4bacd03b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:04.707815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0b3364f6-273c-4093-b3b0-987f4bacd03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:04.765361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3364f6-273c-4093-b3b0-987f4bacd03b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:04.768922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdeda20-f4a1-4b1e-965f-8193760aed3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:04.769557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeda20-f4a1-4b1e-965f-8193760aed3d service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:04.769835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeda20-f4a1-4b1e-965f-8193760aed3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:04.770106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeda20-f4a1-4b1e-965f-8193760aed3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:04.778631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdeda20-f4a1-4b1e-965f-8193760aed3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:04.780060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdeda20-f4a1-4b1e-965f-8193760aed3d service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:04.789389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1049/2095] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:05.594677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cfe8e-addf-4d4c-ab86-b975cf2ae560 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:05.596079 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-081cfe8e-addf-4d4c-ab86-b975cf2ae560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:05.617792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:05.662250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cfe8e-addf-4d4c-ab86-b975cf2ae560 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:05.668578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:05.669462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:05.669743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:05.670048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] PATH_INFO: `/v3/users/6fd87993dc934c80aff8b1563227e4a0` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:05.671247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd87993dc934c80aff8b1563227e4a0)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:05.673239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:05.723677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd87993dc934c80aff8b1563227e4a0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:05.724361 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd87993dc934c80aff8b1563227e4a0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:05.724979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd87993dc934c80aff8b1563227e4a0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:05.732881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd87993dc934c80aff8b1563227e4a0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:05.748160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd87993dc934c80aff8b1563227e4a0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:05.762611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1047/2096] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:05 2022] DELETE /identity/v3/users/6fd87993dc934c80aff8b1563227e4a0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:05.767087 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85d84d-379a-4dd7-8cb4-44cc8da304b7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:05.767534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85d84d-379a-4dd7-8cb4-44cc8da304b7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:05.767683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85d84d-379a-4dd7-8cb4-44cc8da304b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:05.786380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5c85d84d-379a-4dd7-8cb4-44cc8da304b7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:05.787645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5c85d84d-379a-4dd7-8cb4-44cc8da304b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:05.839439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:05.849447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1050/2097] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:05.858939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:05.860364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:05.916101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b143a36-d7fc-48d0-9c22-677b680bc068 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:05.919728 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d5df1-0a88-4679-b216-77f5cda92388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:05.920406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d5df1-0a88-4679-b216-77f5cda92388 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:05.920688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d5df1-0a88-4679-b216-77f5cda92388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:05.920963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d5df1-0a88-4679-b216-77f5cda92388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:05.930157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d5df1-0a88-4679-b216-77f5cda92388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:05.931714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d5df1-0a88-4679-b216-77f5cda92388 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:05.941767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1048/2098] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:06.390085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85d84d-379a-4dd7-8cb4-44cc8da304b7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:06.392485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5c85d84d-379a-4dd7-8cb4-44cc8da304b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:06.450535 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85d84d-379a-4dd7-8cb4-44cc8da304b7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:06.477072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:06.479988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:06.479988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:06.480326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] PATH_INFO: `/v3/projects/2443ea7c59e14e6e8eba6d63f4b7936e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:06.482054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2443ea7c59e14e6e8eba6d63f4b7936e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:06.484567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:06.549807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2443ea7c59e14e6e8eba6d63f4b7936e', 'request_id': 'req-f784bf67-d8b7-4c01-afb5-a9f596bfb252'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:06.559758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2443ea7c59e14e6e8eba6d63f4b7936e', 'request_id': 'req-f784bf67-d8b7-4c01-afb5-a9f596bfb252'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:06.576811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2443ea7c59e14e6e8eba6d63f4b7936e', 'request_id': 'req-f784bf67-d8b7-4c01-afb5-a9f596bfb252'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:06.598347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] The token cache is being invalidate because project 2443ea7c59e14e6e8eba6d63f4b7936e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:06.598867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:06.599715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f784bf67-d8b7-4c01-afb5-a9f596bfb252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:06.604688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1051/2099] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:06 2022] DELETE /identity/v3/projects/2443ea7c59e14e6e8eba6d63f4b7936e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:08.120883 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d5df1-0a88-4679-b216-77f5cda92388 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:08.122370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-102d5df1-0a88-4679-b216-77f5cda92388 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:08.135298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d5df1-0a88-4679-b216-77f5cda92388 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:08.183767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:08.207055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:08.208223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:08.208605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:08.208986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] PATH_INFO: `/v3/users/7dce02643e89494db02134957ba8407e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:08.210482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dce02643e89494db02134957ba8407e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:08.212912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:08.271205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dce02643e89494db02134957ba8407e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:08.271982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dce02643e89494db02134957ba8407e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:08.272956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dce02643e89494db02134957ba8407e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:08.283436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dce02643e89494db02134957ba8407e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:08.305302 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dce02643e89494db02134957ba8407e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:08.338875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1049/2100] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:08 2022] DELETE /identity/v3/users/7dce02643e89494db02134957ba8407e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:08.340600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a1d6-d65a-4b1e-84e6-07a027c36db1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:08.341103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a1d6-d65a-4b1e-84e6-07a027c36db1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:08.341349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a1d6-d65a-4b1e-84e6-07a027c36db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:08.383083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b200a1d6-d65a-4b1e-84e6-07a027c36db1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:08.384553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b200a1d6-d65a-4b1e-84e6-07a027c36db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:08.449309 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1052/2101] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:08.459485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:08.466042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:08.540661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e6ef6-3d62-49f4-94e9-f131d29466f7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:08.545133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4278f-ea55-495e-9ada-1831905707ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:08.545790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4278f-ea55-495e-9ada-1831905707ec service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:08.546078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4278f-ea55-495e-9ada-1831905707ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:08.546353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4278f-ea55-495e-9ada-1831905707ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:08.557110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4278f-ea55-495e-9ada-1831905707ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:08.558552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4278f-ea55-495e-9ada-1831905707ec service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:08.568248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1050/2102] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:35:08.851949 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200a1d6-d65a-4b1e-84e6-07a027c36db1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:08.854399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b200a1d6-d65a-4b1e-84e6-07a027c36db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:08.866962 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200a1d6-d65a-4b1e-84e6-07a027c36db1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:08.872685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:08.873551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:08.873831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:08.874108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] PATH_INFO: `/v3/projects/a50941a4e92849c6a89bcfd6558f5a27` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:08.875355 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a50941a4e92849c6a89bcfd6558f5a27)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:08.877345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:08.932456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a50941a4e92849c6a89bcfd6558f5a27', 'request_id': 'req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:08.940327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a50941a4e92849c6a89bcfd6558f5a27', 'request_id': 'req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:08.949750 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a50941a4e92849c6a89bcfd6558f5a27', 'request_id': 'req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:08.961031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] The token cache is being invalidate because project a50941a4e92849c6a89bcfd6558f5a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:08.961322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:08.962877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e0d85c2b-ea30-4574-8c5e-84e7f3a152fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:08.964477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1053/2103] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:08 2022] DELETE /identity/v3/projects/a50941a4e92849c6a89bcfd6558f5a27 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:35:08.970571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976c5c9-9c23-4b5e-b607-c3e507403075 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:08.970818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976c5c9-9c23-4b5e-b607-c3e507403075 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:08.971118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976c5c9-9c23-4b5e-b607-c3e507403075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:08.988517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6976c5c9-9c23-4b5e-b607-c3e507403075 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:08.989699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6976c5c9-9c23-4b5e-b607-c3e507403075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:09.051398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1051/2104] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:09.059573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f462376-b1ad-40bf-ba5d-2739a9a95ced None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:09.059862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f462376-b1ad-40bf-ba5d-2739a9a95ced None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:09.060142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f462376-b1ad-40bf-ba5d-2739a9a95ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:09.074319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:09.080951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3f462376-b1ad-40bf-ba5d-2739a9a95ced None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:09.082654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3f462376-b1ad-40bf-ba5d-2739a9a95ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:09.091151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1054/2105] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:09.095303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976c5c9-9c23-4b5e-b607-c3e507403075 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:09.101269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:09.105581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976c5c9-9c23-4b5e-b607-c3e507403075 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:09.111364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e724c8-76a5-4623-9321-b51bb8c3dbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:09.112200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e724c8-76a5-4623-9321-b51bb8c3dbce admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:09.112525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e724c8-76a5-4623-9321-b51bb8c3dbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:09.112806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e724c8-76a5-4623-9321-b51bb8c3dbce admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:09.113532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e724c8-76a5-4623-9321-b51bb8c3dbce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:35:09.113837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e724c8-76a5-4623-9321-b51bb8c3dbce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:09.115437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e724c8-76a5-4623-9321-b51bb8c3dbce admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:09.127750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1052/2106] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:35:09.131935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f462376-b1ad-40bf-ba5d-2739a9a95ced None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:09.141763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f462376-b1ad-40bf-ba5d-2739a9a95ced None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:09.146463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a292b0-ea8d-4000-abf7-f9f3e605e889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:09.147104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a292b0-ea8d-4000-abf7-f9f3e605e889 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:09.147389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a292b0-ea8d-4000-abf7-f9f3e605e889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:09.147663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a292b0-ea8d-4000-abf7-f9f3e605e889 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:09.148396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a292b0-ea8d-4000-abf7-f9f3e605e889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:09.150436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a292b0-ea8d-4000-abf7-f9f3e605e889 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:09.175078 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1055/2107] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:09 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:09.179223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e724c8-76a5-4623-9321-b51bb8c3dbce admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:09.189609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e724c8-76a5-4623-9321-b51bb8c3dbce admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:09.250380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba15e5-18af-4889-8663-5bc72f117d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:09.250961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba15e5-18af-4889-8663-5bc72f117d7a admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:09.251242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba15e5-18af-4889-8663-5bc72f117d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:09.251518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba15e5-18af-4889-8663-5bc72f117d7a admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:09.252191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba15e5-18af-4889-8663-5bc72f117d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:09.254137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba15e5-18af-4889-8663-5bc72f117d7a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:09.274198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1053/2108] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:09 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:09.285340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a292b0-ea8d-4000-abf7-f9f3e605e889 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:09.295857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a292b0-ea8d-4000-abf7-f9f3e605e889 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:09.300600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a687ff2-0d43-4339-bf09-7b4b8374229f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:09.301275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687ff2-0d43-4339-bf09-7b4b8374229f admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:09.301563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687ff2-0d43-4339-bf09-7b4b8374229f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:09.301842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687ff2-0d43-4339-bf09-7b4b8374229f admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:09.302582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a687ff2-0d43-4339-bf09-7b4b8374229f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:09.304257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a687ff2-0d43-4339-bf09-7b4b8374229f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:09.319518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1056/2109] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:09.320832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba15e5-18af-4889-8663-5bc72f117d7a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:09.333489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba15e5-18af-4889-8663-5bc72f117d7a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:09.339194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15955ff0-3a0b-4a12-a702-7d922f358701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:09.339887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955ff0-3a0b-4a12-a702-7d922f358701 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:09.340470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955ff0-3a0b-4a12-a702-7d922f358701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:09.340814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15955ff0-3a0b-4a12-a702-7d922f358701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:09.341797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15955ff0-3a0b-4a12-a702-7d922f358701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:09.343574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15955ff0-3a0b-4a12-a702-7d922f358701 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:09.355093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1054/2110] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:09.358961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a687ff2-0d43-4339-bf09-7b4b8374229f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:09.369933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a687ff2-0d43-4339-bf09-7b4b8374229f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:09.375673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f3604-9383-497e-9991-cf7ab341b8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:09.377599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f3604-9383-497e-9991-cf7ab341b8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:09.377952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f3604-9383-497e-9991-cf7ab341b8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:09.378330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f3604-9383-497e-9991-cf7ab341b8b6 admin admin] PATH_INFO: `/v3/projects/a61e8cc1224e4aec9a77997ce2aa8483/users/963ac179d0f845e794e41f3e97dd444b/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:09.405527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f3604-9383-497e-9991-cf7ab341b8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61e8cc1224e4aec9a77997ce2aa8483, user_id=963ac179d0f845e794e41f3e97dd444b, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:09.409944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f3604-9383-497e-9991-cf7ab341b8b6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:09.432572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1057/2111] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:09 2022] PUT /identity/v3/projects/a61e8cc1224e4aec9a77997ce2aa8483/users/963ac179d0f845e794e41f3e97dd444b/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:09.445028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526cb85-8272-49bb-8c5c-cf0fdea6fba9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:09.445667 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526cb85-8272-49bb-8c5c-cf0fdea6fba9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:09.446163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526cb85-8272-49bb-8c5c-cf0fdea6fba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:09.507574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a526cb85-8272-49bb-8c5c-cf0fdea6fba9 None None] MFA Rules not processed for user `963ac179d0f845e794e41f3e97dd444b`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:09.508970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a526cb85-8272-49bb-8c5c-cf0fdea6fba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:09.618044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1055/2112] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:35:09.629273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f3604-9383-497e-9991-cf7ab341b8b6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:09.631600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6b1f3604-9383-497e-9991-cf7ab341b8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:09.705641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f3604-9383-497e-9991-cf7ab341b8b6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:09.709454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b488a6c-4e28-41e7-bd09-7439054a1510 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:09.709934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b488a6c-4e28-41e7-bd09-7439054a1510 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:09.710213 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b488a6c-4e28-41e7-bd09-7439054a1510 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:09.710488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b488a6c-4e28-41e7-bd09-7439054a1510 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:09.719347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b488a6c-4e28-41e7-bd09-7439054a1510 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:09.720807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b488a6c-4e28-41e7-bd09-7439054a1510 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:09.737825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1058/2113] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:11.053970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526cb85-8272-49bb-8c5c-cf0fdea6fba9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:11.055375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a526cb85-8272-49bb-8c5c-cf0fdea6fba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:11.125682 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526cb85-8272-49bb-8c5c-cf0fdea6fba9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:11.130630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:11.131546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:11.131828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:11.132106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] PATH_INFO: `/v3/users/5afe7402a6fb42748b77ba01853b4b54` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:11.133411 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5afe7402a6fb42748b77ba01853b4b54)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:11.135576 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:11.137274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:11.188142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5afe7402a6fb42748b77ba01853b4b54'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:11.188811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5afe7402a6fb42748b77ba01853b4b54'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:11.189422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5afe7402a6fb42748b77ba01853b4b54'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:11.197225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5afe7402a6fb42748b77ba01853b4b54'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:11.212824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5afe7402a6fb42748b77ba01853b4b54'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:11.230075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1056/2114] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:11 2022] DELETE /identity/v3/users/5afe7402a6fb42748b77ba01853b4b54 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:35:11.235647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f709051-ef90-430a-a5c1-e7d330e40b2f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:11.236102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f709051-ef90-430a-a5c1-e7d330e40b2f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:11.236195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f709051-ef90-430a-a5c1-e7d330e40b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:11.259903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5f709051-ef90-430a-a5c1-e7d330e40b2f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:11.261780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5f709051-ef90-430a-a5c1-e7d330e40b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:11.341539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1059/2115] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:11.352410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:11.354820 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:11.446691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ffdee-9f9e-4426-a8a4-bb4744511a62 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:11.450804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd443a9f-62f9-4ab2-9bd0-51280bd4ddec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:11.451456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd443a9f-62f9-4ab2-9bd0-51280bd4ddec service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:11.451739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd443a9f-62f9-4ab2-9bd0-51280bd4ddec service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:11.452137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd443a9f-62f9-4ab2-9bd0-51280bd4ddec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:11.461594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd443a9f-62f9-4ab2-9bd0-51280bd4ddec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:11.463612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd443a9f-62f9-4ab2-9bd0-51280bd4ddec service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:11.473632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1057/2116] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:35:11.770308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f709051-ef90-430a-a5c1-e7d330e40b2f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:11.771821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5f709051-ef90-430a-a5c1-e7d330e40b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:11.794165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f709051-ef90-430a-a5c1-e7d330e40b2f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:11.809701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:11.811184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:11.811464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:11.811899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] PATH_INFO: `/v3/projects/7da55d6136d549748e95617caa09da14` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:11.813235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7da55d6136d549748e95617caa09da14)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:11.815273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:11.863108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:11.875028 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7da55d6136d549748e95617caa09da14', 'request_id': 'req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:11.883528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7da55d6136d549748e95617caa09da14', 'request_id': 'req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:11.895495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7da55d6136d549748e95617caa09da14', 'request_id': 'req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:11.910436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] The token cache is being invalidate because project 7da55d6136d549748e95617caa09da14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:11.910844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:11.911418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7823e1ce-df1d-4b61-a1f3-c0d3aec50791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:11.913907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1060/2117] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:11 2022] DELETE /identity/v3/projects/7da55d6136d549748e95617caa09da14 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:11.920099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b7912-5ab3-44a1-bd71-c7e60f2934dc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:11.920416 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b7912-5ab3-44a1-bd71-c7e60f2934dc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:11.920697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b7912-5ab3-44a1-bd71-c7e60f2934dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:11.941359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ce4b7912-5ab3-44a1-bd71-c7e60f2934dc None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:11.942654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ce4b7912-5ab3-44a1-bd71-c7e60f2934dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:12.011254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1058/2118] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:12.018999 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e7c01-20c2-4c7c-b9e5-7c111996eb9b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:12.019289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e7c01-20c2-4c7c-b9e5-7c111996eb9b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:12.019568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e7c01-20c2-4c7c-b9e5-7c111996eb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:12.044116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-536e7c01-20c2-4c7c-b9e5-7c111996eb9b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:12.046024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-536e7c01-20c2-4c7c-b9e5-7c111996eb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:12.056558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1061/2119] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:12.061495 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b7912-5ab3-44a1-bd71-c7e60f2934dc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:12.073795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b7912-5ab3-44a1-bd71-c7e60f2934dc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:12.078553 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918313e-cd45-494f-b39e-b2c43fc89fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:12.079222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918313e-cd45-494f-b39e-b2c43fc89fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:12.079506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918313e-cd45-494f-b39e-b2c43fc89fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:12.079788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918313e-cd45-494f-b39e-b2c43fc89fb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:12.080551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918313e-cd45-494f-b39e-b2c43fc89fb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:35:12.080860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918313e-cd45-494f-b39e-b2c43fc89fb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:12.082467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918313e-cd45-494f-b39e-b2c43fc89fb0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:12.102203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1059/2120] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:35:12.107714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e7c01-20c2-4c7c-b9e5-7c111996eb9b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:12.117511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e7c01-20c2-4c7c-b9e5-7c111996eb9b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:12.122595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f4c30-660f-4389-8486-4931a298b72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:12.123285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f4c30-660f-4389-8486-4931a298b72a admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:12.123565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f4c30-660f-4389-8486-4931a298b72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:12.123845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f4c30-660f-4389-8486-4931a298b72a admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:12.124577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f4c30-660f-4389-8486-4931a298b72a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:12.126471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f4c30-660f-4389-8486-4931a298b72a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:12.154340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1062/2121] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:12 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:12.158408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918313e-cd45-494f-b39e-b2c43fc89fb0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:12.169273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918313e-cd45-494f-b39e-b2c43fc89fb0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:12.174609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:12.241656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479f743-15fc-47c6-94e2-cb789972f53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:12.243152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f743-15fc-47c6-94e2-cb789972f53c admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:12.243152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f743-15fc-47c6-94e2-cb789972f53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:12.243152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f743-15fc-47c6-94e2-cb789972f53c admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:12.244895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479f743-15fc-47c6-94e2-cb789972f53c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:12.247166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479f743-15fc-47c6-94e2-cb789972f53c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:12.274699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1060/2122] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:12 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:12.276165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f4c30-660f-4389-8486-4931a298b72a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:12.287779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f4c30-660f-4389-8486-4931a298b72a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:12.294435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc3c05-6f55-47e5-938f-11496612a1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:12.295189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc3c05-6f55-47e5-938f-11496612a1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:12.295537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc3c05-6f55-47e5-938f-11496612a1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:12.295879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc3c05-6f55-47e5-938f-11496612a1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:12.296739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc3c05-6f55-47e5-938f-11496612a1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:12.298409 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc3c05-6f55-47e5-938f-11496612a1c1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:12.315800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1063/2123] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:35:12.319803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479f743-15fc-47c6-94e2-cb789972f53c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:12.332163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479f743-15fc-47c6-94e2-cb789972f53c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:12.337520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56fc92-27ad-4142-a81a-2fff8767b457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:12.339477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56fc92-27ad-4142-a81a-2fff8767b457 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:12.340244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56fc92-27ad-4142-a81a-2fff8767b457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:12.340244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56fc92-27ad-4142-a81a-2fff8767b457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:12.341092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56fc92-27ad-4142-a81a-2fff8767b457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:12.342836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56fc92-27ad-4142-a81a-2fff8767b457 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:12.359436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1061/2124] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:12.361376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc3c05-6f55-47e5-938f-11496612a1c1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:12.372507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc3c05-6f55-47e5-938f-11496612a1c1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:12.377862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487f54a-54de-4dcf-896b-c40dab8d6d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:12.378985 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487f54a-54de-4dcf-896b-c40dab8d6d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:12.379361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487f54a-54de-4dcf-896b-c40dab8d6d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:12.379839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487f54a-54de-4dcf-896b-c40dab8d6d81 admin admin] PATH_INFO: `/v3/projects/88dd2b2db7b846ff99d3091575fa0b04/users/626ba8ee94d24de1839d4f3471bbfe8f/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:12.397873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487f54a-54de-4dcf-896b-c40dab8d6d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88dd2b2db7b846ff99d3091575fa0b04, user_id=626ba8ee94d24de1839d4f3471bbfe8f, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:12.401854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487f54a-54de-4dcf-896b-c40dab8d6d81 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:12.415970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1064/2125] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:12 2022] PUT /identity/v3/projects/88dd2b2db7b846ff99d3091575fa0b04/users/626ba8ee94d24de1839d4f3471bbfe8f/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:35:12.426972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37079f5a-54d2-4a29-90d0-4369d9f5c8d8 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:12.427263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37079f5a-54d2-4a29-90d0-4369d9f5c8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:12.427540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37079f5a-54d2-4a29-90d0-4369d9f5c8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:12.473177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-37079f5a-54d2-4a29-90d0-4369d9f5c8d8 None None] MFA Rules not processed for user `626ba8ee94d24de1839d4f3471bbfe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:12.474566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-37079f5a-54d2-4a29-90d0-4369d9f5c8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:12.550561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1062/2126] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:12.565132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487f54a-54de-4dcf-896b-c40dab8d6d81 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:12.567586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4487f54a-54de-4dcf-896b-c40dab8d6d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:12.648695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487f54a-54de-4dcf-896b-c40dab8d6d81 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:12.655979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d7e83-5781-40cf-9e16-d42c62460505 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e07c59babad479cadbf74f15ee5eea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:12.656815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d7e83-5781-40cf-9e16-d42c62460505 service glance] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:12.657164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d7e83-5781-40cf-9e16-d42c62460505 service glance] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:12.657544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d7e83-5781-40cf-9e16-d42c62460505 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:12.672746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d7e83-5781-40cf-9e16-d42c62460505 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:12.674447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d7e83-5781-40cf-9e16-d42c62460505 service glance] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:12.688478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1065/2127] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:14.881514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37079f5a-54d2-4a29-90d0-4369d9f5c8d8 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:14.885759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-37079f5a-54d2-4a29-90d0-4369d9f5c8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:14.969780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37079f5a-54d2-4a29-90d0-4369d9f5c8d8 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:14.975342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:14.976208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:14.976602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:14.976804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] PATH_INFO: `/v3/users/c5e836103d274f9386f2dd933acd982d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:14.978056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5e836103d274f9386f2dd933acd982d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:14.980015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:15.004075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:15.030921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5e836103d274f9386f2dd933acd982d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:15.031629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5e836103d274f9386f2dd933acd982d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:15.032235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5e836103d274f9386f2dd933acd982d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:15.039830 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5e836103d274f9386f2dd933acd982d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:15.053478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ffa69218-164f-4804-b481-cfb4f7aa33f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5e836103d274f9386f2dd933acd982d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:15.070791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1063/2128] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:14 2022] DELETE /identity/v3/users/c5e836103d274f9386f2dd933acd982d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:15.255380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d7e83-5781-40cf-9e16-d42c62460505 service glance] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:15.269198 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d7e83-5781-40cf-9e16-d42c62460505 service glance] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:15.345616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:15.346397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:15.346680 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:15.346955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] PATH_INFO: `/v3/projects/f52012c88f1e4cc59092d37fcc0fa23a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:15.348144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f52012c88f1e4cc59092d37fcc0fa23a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:15.350176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:15.364812 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:15.403439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f52012c88f1e4cc59092d37fcc0fa23a', 'request_id': 'req-249cb9bd-93be-4ae9-807d-a73e82e6ef03'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:15.409991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f52012c88f1e4cc59092d37fcc0fa23a', 'request_id': 'req-249cb9bd-93be-4ae9-807d-a73e82e6ef03'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:15.417558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f52012c88f1e4cc59092d37fcc0fa23a', 'request_id': 'req-249cb9bd-93be-4ae9-807d-a73e82e6ef03'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:15.427807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] The token cache is being invalidate because project f52012c88f1e4cc59092d37fcc0fa23a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:15.428099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:15.428755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-249cb9bd-93be-4ae9-807d-a73e82e6ef03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:15.430304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1066/2129] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:15 2022] DELETE /identity/v3/projects/f52012c88f1e4cc59092d37fcc0fa23a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:15.438102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a7f0d-451e-4c78-b11d-e963f99f841f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:15.438419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a7f0d-451e-4c78-b11d-e963f99f841f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:15.438704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a7f0d-451e-4c78-b11d-e963f99f841f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:15.459014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-458a7f0d-451e-4c78-b11d-e963f99f841f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:15.460230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-458a7f0d-451e-4c78-b11d-e963f99f841f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:15.523003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1064/2130] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:15.533111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa85ee7-b8e2-480c-b613-4bc71cdb41a1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:15.533397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa85ee7-b8e2-480c-b613-4bc71cdb41a1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:15.533676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa85ee7-b8e2-480c-b613-4bc71cdb41a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:15.552608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-efa85ee7-b8e2-480c-b613-4bc71cdb41a1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:15.553815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-efa85ee7-b8e2-480c-b613-4bc71cdb41a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:15.562228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1067/2131] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:15.566288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a7f0d-451e-4c78-b11d-e963f99f841f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:15.577107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a7f0d-451e-4c78-b11d-e963f99f841f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:15.581836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be1071-0c91-41a9-83bf-8d45cef7c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:15.582505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be1071-0c91-41a9-83bf-8d45cef7c50c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:15.582791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be1071-0c91-41a9-83bf-8d45cef7c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:15.583071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be1071-0c91-41a9-83bf-8d45cef7c50c admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:15.583807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be1071-0c91-41a9-83bf-8d45cef7c50c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:35:15.584120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be1071-0c91-41a9-83bf-8d45cef7c50c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:15.585758 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be1071-0c91-41a9-83bf-8d45cef7c50c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:15.602021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1065/2132] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:35:15.602937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa85ee7-b8e2-480c-b613-4bc71cdb41a1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:15.612700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa85ee7-b8e2-480c-b613-4bc71cdb41a1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:15.617793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe2b50-7d02-4605-9a85-1a5d1c1a41dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:15.618454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe2b50-7d02-4605-9a85-1a5d1c1a41dd admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:15.618739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe2b50-7d02-4605-9a85-1a5d1c1a41dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:15.619046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe2b50-7d02-4605-9a85-1a5d1c1a41dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:15.620082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe2b50-7d02-4605-9a85-1a5d1c1a41dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:15.622026 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe2b50-7d02-4605-9a85-1a5d1c1a41dd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:15.654822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1068/2133] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:15 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:15.656119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be1071-0c91-41a9-83bf-8d45cef7c50c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:15.667036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be1071-0c91-41a9-83bf-8d45cef7c50c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:15.750279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4480ce-5e36-46e1-92f0-2f4be3d26b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:15.750986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4480ce-5e36-46e1-92f0-2f4be3d26b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:15.751273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4480ce-5e36-46e1-92f0-2f4be3d26b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:15.751554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4480ce-5e36-46e1-92f0-2f4be3d26b1d admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:15.752270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4480ce-5e36-46e1-92f0-2f4be3d26b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:15.754276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4480ce-5e36-46e1-92f0-2f4be3d26b1d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:15.775638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1066/2134] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:15 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:15.779683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe2b50-7d02-4605-9a85-1a5d1c1a41dd admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:15.790179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe2b50-7d02-4605-9a85-1a5d1c1a41dd admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:15.795085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a6cd1-1401-43fb-99cf-bc5a9820f3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:15.795744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a6cd1-1401-43fb-99cf-bc5a9820f3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:15.796030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a6cd1-1401-43fb-99cf-bc5a9820f3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:15.796380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a6cd1-1401-43fb-99cf-bc5a9820f3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:15.797122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a6cd1-1401-43fb-99cf-bc5a9820f3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:15.798735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a6cd1-1401-43fb-99cf-bc5a9820f3a3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:15.809708 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1069/2135] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:15.813708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4480ce-5e36-46e1-92f0-2f4be3d26b1d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:15.824244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4480ce-5e36-46e1-92f0-2f4be3d26b1d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:15.828950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4effb3f-f67c-4cec-b80f-7b53b9163474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:15.829613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4effb3f-f67c-4cec-b80f-7b53b9163474 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:15.829898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4effb3f-f67c-4cec-b80f-7b53b9163474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:15.830177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4effb3f-f67c-4cec-b80f-7b53b9163474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:15.830904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4effb3f-f67c-4cec-b80f-7b53b9163474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:15.832524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4effb3f-f67c-4cec-b80f-7b53b9163474 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:15.842839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1067/2136] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:15.846752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a6cd1-1401-43fb-99cf-bc5a9820f3a3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:15.857001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a6cd1-1401-43fb-99cf-bc5a9820f3a3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:15.861779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77416173-0e70-488f-b1e0-bd82752f7de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:15.862668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416173-0e70-488f-b1e0-bd82752f7de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:15.862947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416173-0e70-488f-b1e0-bd82752f7de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:15.863220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416173-0e70-488f-b1e0-bd82752f7de6 admin admin] PATH_INFO: `/v3/projects/4d2c3155d45b453c8ba8294b3f60edd2/users/4daca41051d74b4eb229f947e3cf314f/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:15.878496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77416173-0e70-488f-b1e0-bd82752f7de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2c3155d45b453c8ba8294b3f60edd2, user_id=4daca41051d74b4eb229f947e3cf314f, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:15.881191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77416173-0e70-488f-b1e0-bd82752f7de6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:15.896472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1070/2137] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:15 2022] PUT /identity/v3/projects/4d2c3155d45b453c8ba8294b3f60edd2/users/4daca41051d74b4eb229f947e3cf314f/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:15.906952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793191e2-6ee5-47ef-8554-d5bdd8490622 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:15.907473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793191e2-6ee5-47ef-8554-d5bdd8490622 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:15.907729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793191e2-6ee5-47ef-8554-d5bdd8490622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:15.914821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:15.950013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-793191e2-6ee5-47ef-8554-d5bdd8490622 None None] MFA Rules not processed for user `4daca41051d74b4eb229f947e3cf314f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:15.951249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-793191e2-6ee5-47ef-8554-d5bdd8490622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:16.029625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1068/2138] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:16.035313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77416173-0e70-488f-b1e0-bd82752f7de6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.109775 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77416173-0e70-488f-b1e0-bd82752f7de6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.114333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfb9a3-f313-4463-b9eb-af3b3039a0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.114988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfb9a3-f313-4463-b9eb-af3b3039a0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.115272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfb9a3-f313-4463-b9eb-af3b3039a0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.115547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfb9a3-f313-4463-b9eb-af3b3039a0d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.116233 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfb9a3-f313-4463-b9eb-af3b3039a0d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.118186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfb9a3-f313-4463-b9eb-af3b3039a0d7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.143670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1071/2139] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:16 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:16.147703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793191e2-6ee5-47ef-8554-d5bdd8490622 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.158597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793191e2-6ee5-47ef-8554-d5bdd8490622 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.219670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b0140-9c23-4513-9b1e-6858cfd496a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.220367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b0140-9c23-4513-9b1e-6858cfd496a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.220650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b0140-9c23-4513-9b1e-6858cfd496a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.220921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b0140-9c23-4513-9b1e-6858cfd496a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.221603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b0140-9c23-4513-9b1e-6858cfd496a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.223515 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b0140-9c23-4513-9b1e-6858cfd496a2 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.242891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1069/2140] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:16 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:16.247513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfb9a3-f313-4463-b9eb-af3b3039a0d7 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.257637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfb9a3-f313-4463-b9eb-af3b3039a0d7 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.262482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ba669-03b1-405b-a398-f7ecf10e4811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.263148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ba669-03b1-405b-a398-f7ecf10e4811 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.263429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ba669-03b1-405b-a398-f7ecf10e4811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.263702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ba669-03b1-405b-a398-f7ecf10e4811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.264501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ba669-03b1-405b-a398-f7ecf10e4811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.266132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ba669-03b1-405b-a398-f7ecf10e4811 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.277460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1072/2141] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:16.281552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b0140-9c23-4513-9b1e-6858cfd496a2 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.292485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b0140-9c23-4513-9b1e-6858cfd496a2 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.297281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe8e36-16bd-4871-b6f3-a5ab85f25c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.297979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe8e36-16bd-4871-b6f3-a5ab85f25c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.298260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe8e36-16bd-4871-b6f3-a5ab85f25c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.298541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe8e36-16bd-4871-b6f3-a5ab85f25c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.299277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe8e36-16bd-4871-b6f3-a5ab85f25c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.300910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe8e36-16bd-4871-b6f3-a5ab85f25c2a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.315151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1070/2142] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:16.319305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ba669-03b1-405b-a398-f7ecf10e4811 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.329739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ba669-03b1-405b-a398-f7ecf10e4811 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.334529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d720a98-63c0-4d17-b8f9-7d317426cddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.335404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d720a98-63c0-4d17-b8f9-7d317426cddc admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.335682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d720a98-63c0-4d17-b8f9-7d317426cddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.335957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d720a98-63c0-4d17-b8f9-7d317426cddc admin admin] PATH_INFO: `/v3/projects/8fecfc501af3473d94260aa9467ad9c4/users/4dbeacedf1924c85a199750def804634/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.339191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:16.352564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d720a98-63c0-4d17-b8f9-7d317426cddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fecfc501af3473d94260aa9467ad9c4, user_id=4dbeacedf1924c85a199750def804634, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.355778 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d720a98-63c0-4d17-b8f9-7d317426cddc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.372603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1073/2143] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:16 2022] PUT /identity/v3/projects/8fecfc501af3473d94260aa9467ad9c4/users/4dbeacedf1924c85a199750def804634/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:16.382614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab223d-ebf8-4df6-867f-e63456e09895 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.382981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab223d-ebf8-4df6-867f-e63456e09895 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.383270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab223d-ebf8-4df6-867f-e63456e09895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.408887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-79ab223d-ebf8-4df6-867f-e63456e09895 None None] MFA Rules not processed for user `4dbeacedf1924c85a199750def804634`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:16.410091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-79ab223d-ebf8-4df6-867f-e63456e09895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:16.473211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:16.477216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1071/2144] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:16.482111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d720a98-63c0-4d17-b8f9-7d317426cddc admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.550267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d720a98-63c0-4d17-b8f9-7d317426cddc admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.555184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c9721-381a-47ed-8cca-6ddfcfbdb814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.555842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c9721-381a-47ed-8cca-6ddfcfbdb814 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.556125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c9721-381a-47ed-8cca-6ddfcfbdb814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.556437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c9721-381a-47ed-8cca-6ddfcfbdb814 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.557150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c9721-381a-47ed-8cca-6ddfcfbdb814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.559055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c9721-381a-47ed-8cca-6ddfcfbdb814 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.585124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1074/2145] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:16 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:16.589762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab223d-ebf8-4df6-867f-e63456e09895 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.600289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab223d-ebf8-4df6-867f-e63456e09895 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.658532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95896fc8-5127-44d2-94a9-df3bed1a3e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.659218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95896fc8-5127-44d2-94a9-df3bed1a3e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.659501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95896fc8-5127-44d2-94a9-df3bed1a3e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.659918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95896fc8-5127-44d2-94a9-df3bed1a3e81 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.660652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95896fc8-5127-44d2-94a9-df3bed1a3e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.662666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95896fc8-5127-44d2-94a9-df3bed1a3e81 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.684304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1072/2146] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:16 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:35:16.688834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c9721-381a-47ed-8cca-6ddfcfbdb814 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.701451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c9721-381a-47ed-8cca-6ddfcfbdb814 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.706084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2ef4e-275b-4403-9cb2-83f4c0a1d526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.706740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2ef4e-275b-4403-9cb2-83f4c0a1d526 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.707025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2ef4e-275b-4403-9cb2-83f4c0a1d526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.707306 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2ef4e-275b-4403-9cb2-83f4c0a1d526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.708037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2ef4e-275b-4403-9cb2-83f4c0a1d526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.709772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2ef4e-275b-4403-9cb2-83f4c0a1d526 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.721769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1075/2147] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:16.726006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95896fc8-5127-44d2-94a9-df3bed1a3e81 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.737692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95896fc8-5127-44d2-94a9-df3bed1a3e81 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.743032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b273dd9-9fc2-4c09-b245-fc433a9b161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.744009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b273dd9-9fc2-4c09-b245-fc433a9b161e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.744324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b273dd9-9fc2-4c09-b245-fc433a9b161e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.744944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b273dd9-9fc2-4c09-b245-fc433a9b161e admin admin] PATH_INFO: `/v3/projects/7dbe3ba631124c899cbdda93dd23c3e5/users/aeb843ad7e9748e18ca9de0f5c45f5b6/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.762109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b273dd9-9fc2-4c09-b245-fc433a9b161e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dbe3ba631124c899cbdda93dd23c3e5, user_id=aeb843ad7e9748e18ca9de0f5c45f5b6, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.764799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b273dd9-9fc2-4c09-b245-fc433a9b161e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.779365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1073/2148] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:16 2022] PUT /identity/v3/projects/7dbe3ba631124c899cbdda93dd23c3e5/users/aeb843ad7e9748e18ca9de0f5c45f5b6/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:16.783244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2ef4e-275b-4403-9cb2-83f4c0a1d526 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.793944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2ef4e-275b-4403-9cb2-83f4c0a1d526 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.851617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d31d09-4286-4a60-893f-b86c6f50471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.852288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d31d09-4286-4a60-893f-b86c6f50471b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.852605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d31d09-4286-4a60-893f-b86c6f50471b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.852881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d31d09-4286-4a60-893f-b86c6f50471b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.854632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d31d09-4286-4a60-893f-b86c6f50471b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.856240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d31d09-4286-4a60-893f-b86c6f50471b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.866093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1076/2149] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:16.870357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b273dd9-9fc2-4c09-b245-fc433a9b161e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:16.880440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b273dd9-9fc2-4c09-b245-fc433a9b161e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:16.885715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760760fd-34fa-4e96-b0cb-92bc78830331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:16.886620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760760fd-34fa-4e96-b0cb-92bc78830331 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.886903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760760fd-34fa-4e96-b0cb-92bc78830331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.887214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760760fd-34fa-4e96-b0cb-92bc78830331 admin admin] PATH_INFO: `/v3/projects/7dbe3ba631124c899cbdda93dd23c3e5/users/aeb843ad7e9748e18ca9de0f5c45f5b6/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.889321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760760fd-34fa-4e96-b0cb-92bc78830331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dbe3ba631124c899cbdda93dd23c3e5, user_id=aeb843ad7e9748e18ca9de0f5c45f5b6, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:16.892169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760760fd-34fa-4e96-b0cb-92bc78830331 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:16.903689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1074/2150] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:16 2022] PUT /identity/v3/projects/7dbe3ba631124c899cbdda93dd23c3e5/users/aeb843ad7e9748e18ca9de0f5c45f5b6/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:16.914868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022059-8f02-48d3-b251-8a5111c12db2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:16.915094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022059-8f02-48d3-b251-8a5111c12db2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:16.915379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022059-8f02-48d3-b251-8a5111c12db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:16.944443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9b022059-8f02-48d3-b251-8a5111c12db2 None None] MFA Rules not processed for user `aeb843ad7e9748e18ca9de0f5c45f5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:16.945651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9b022059-8f02-48d3-b251-8a5111c12db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:16.989563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:17.018378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1077/2151] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:17.030196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760760fd-34fa-4e96-b0cb-92bc78830331 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:17.035823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-760760fd-34fa-4e96-b0cb-92bc78830331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:17.093900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760760fd-34fa-4e96-b0cb-92bc78830331 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:17.097761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee899dc8-8210-4e7e-89df-59069a036139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:17.098403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee899dc8-8210-4e7e-89df-59069a036139 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:17.098716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee899dc8-8210-4e7e-89df-59069a036139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:17.098993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee899dc8-8210-4e7e-89df-59069a036139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:17.108181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee899dc8-8210-4e7e-89df-59069a036139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:17.109643 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee899dc8-8210-4e7e-89df-59069a036139 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:17.119192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1075/2152] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:35:17.636615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b022059-8f02-48d3-b251-8a5111c12db2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:17.638012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9b022059-8f02-48d3-b251-8a5111c12db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:17.649707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b022059-8f02-48d3-b251-8a5111c12db2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:17.653516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91ece9-2614-4373-8c8f-8653adf9f45a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:17.654227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ece9-2614-4373-8c8f-8653adf9f45a service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:17.654524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ece9-2614-4373-8c8f-8653adf9f45a service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:17.654801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ece9-2614-4373-8c8f-8653adf9f45a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:17.715522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91ece9-2614-4373-8c8f-8653adf9f45a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:17.716902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91ece9-2614-4373-8c8f-8653adf9f45a service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:17.725804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1078/2153] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:25.971759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee899dc8-8210-4e7e-89df-59069a036139 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:25.981175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee899dc8-8210-4e7e-89df-59069a036139 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:25.985528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832619e-2170-47dc-9c83-85a4279ea613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:25.986221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832619e-2170-47dc-9c83-85a4279ea613 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:25.986565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832619e-2170-47dc-9c83-85a4279ea613 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:25.986803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832619e-2170-47dc-9c83-85a4279ea613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:26.046641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:26.051295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832619e-2170-47dc-9c83-85a4279ea613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:26.053198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832619e-2170-47dc-9c83-85a4279ea613 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:26.063144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1076/2154] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:27.721668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=37 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:28.222790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91ece9-2614-4373-8c8f-8653adf9f45a service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:28.291571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91ece9-2614-4373-8c8f-8653adf9f45a service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:28.296102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:28.297002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:28.297283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:28.297555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] PATH_INFO: `/v3/users/4daca41051d74b4eb229f947e3cf314f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:28.298729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4daca41051d74b4eb229f947e3cf314f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:28.300703 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:28.346905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4daca41051d74b4eb229f947e3cf314f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:28.354160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4daca41051d74b4eb229f947e3cf314f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:28.354828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4daca41051d74b4eb229f947e3cf314f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:28.366349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4daca41051d74b4eb229f947e3cf314f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:28.373463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4daca41051d74b4eb229f947e3cf314f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:28.379257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1079/2155] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:28 2022] DELETE /identity/v3/users/4daca41051d74b4eb229f947e3cf314f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:28.383777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4f0a0-09b0-4ac3-9b29-a966dcd868ff None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:28.385510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4f0a0-09b0-4ac3-9b29-a966dcd868ff None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:28.385796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4f0a0-09b0-4ac3-9b29-a966dcd868ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:28.403317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-04d4f0a0-09b0-4ac3-9b29-a966dcd868ff None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:28.404483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-04d4f0a0-09b0-4ac3-9b29-a966dcd868ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:28.461287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1077/2156] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:28.471586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:28.473047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:28.530893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258914-b1ea-4237-ae7a-276318c9e966 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:28.534931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026e548-c0dd-49e7-981d-fdb8edc9d957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:28.535590 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026e548-c0dd-49e7-981d-fdb8edc9d957 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:28.535875 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026e548-c0dd-49e7-981d-fdb8edc9d957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:28.536154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026e548-c0dd-49e7-981d-fdb8edc9d957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:28.546345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026e548-c0dd-49e7-981d-fdb8edc9d957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:28.547785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026e548-c0dd-49e7-981d-fdb8edc9d957 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:28.558163 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1080/2157] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:28.845978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4f0a0-09b0-4ac3-9b29-a966dcd868ff None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:28.847465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-04d4f0a0-09b0-4ac3-9b29-a966dcd868ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:28.861476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4f0a0-09b0-4ac3-9b29-a966dcd868ff None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:28.866743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:28.867625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:28.868054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:28.868363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] PATH_INFO: `/v3/projects/4d2c3155d45b453c8ba8294b3f60edd2` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:28.870134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d2c3155d45b453c8ba8294b3f60edd2)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:28.872069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:28.933813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d2c3155d45b453c8ba8294b3f60edd2', 'request_id': 'req-618b87d4-e2cc-484d-9eff-2246d96a5a63'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:28.943278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d2c3155d45b453c8ba8294b3f60edd2', 'request_id': 'req-618b87d4-e2cc-484d-9eff-2246d96a5a63'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:28.949670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d2c3155d45b453c8ba8294b3f60edd2', 'request_id': 'req-618b87d4-e2cc-484d-9eff-2246d96a5a63'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:28.970085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] The token cache is being invalidate because project 4d2c3155d45b453c8ba8294b3f60edd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:28.970472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:28.971332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:28.975849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1078/2158] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:28 2022] DELETE /identity/v3/projects/4d2c3155d45b453c8ba8294b3f60edd2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:28.977136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026e548-c0dd-49e7-981d-fdb8edc9d957 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:28.989864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026e548-c0dd-49e7-981d-fdb8edc9d957 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:29.020107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:29.060760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:29.061644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:29.061928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:29.062208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] PATH_INFO: `/v3/users/4dbeacedf1924c85a199750def804634` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:29.063399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dbeacedf1924c85a199750def804634)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:29.065445 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:29.111972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dbeacedf1924c85a199750def804634'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.119665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dbeacedf1924c85a199750def804634'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.120505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dbeacedf1924c85a199750def804634'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.137417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dbeacedf1924c85a199750def804634'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.145152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dbeacedf1924c85a199750def804634'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.151479 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1081/2159] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:28 2022] DELETE /identity/v3/users/4dbeacedf1924c85a199750def804634 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:29.289864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:29.292181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:29.308244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b87d4-e2cc-484d-9eff-2246d96a5a63 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:29.370519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:29.371402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:29.371687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:29.371970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] PATH_INFO: `/v3/users/963ac179d0f845e794e41f3e97dd444b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:29.373203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=963ac179d0f845e794e41f3e97dd444b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:29.375139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:29.398651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:29.427393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963ac179d0f845e794e41f3e97dd444b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.428073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963ac179d0f845e794e41f3e97dd444b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.428835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963ac179d0f845e794e41f3e97dd444b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.430412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:29.434658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:29.442282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963ac179d0f845e794e41f3e97dd444b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.451307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c2c50-6a22-46c1-bd59-1587f9af68eb admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:29.456054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:29.456967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:29.457239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:29.457517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] PATH_INFO: `/v3/projects/8fecfc501af3473d94260aa9467ad9c4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:29.458704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fecfc501af3473d94260aa9467ad9c4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:29.460892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:29.469440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-e6d9469d-0a60-4ede-b33d-9183d7e65dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963ac179d0f845e794e41f3e97dd444b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.483844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1079/2160] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:29 2022] DELETE /identity/v3/users/963ac179d0f845e794e41f3e97dd444b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:29.488923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9509676-966b-420b-bcad-91d3d9adf627 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:29.489328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9509676-966b-420b-bcad-91d3d9adf627 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:29.489712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9509676-966b-420b-bcad-91d3d9adf627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:29.508526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b9509676-966b-420b-bcad-91d3d9adf627 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:29.510557 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b9509676-966b-420b-bcad-91d3d9adf627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:29.514339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fecfc501af3473d94260aa9467ad9c4', 'request_id': 'req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.521570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fecfc501af3473d94260aa9467ad9c4', 'request_id': 'req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.531320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fecfc501af3473d94260aa9467ad9c4', 'request_id': 'req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.545071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] The token cache is being invalidate because project 8fecfc501af3473d94260aa9467ad9c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:29.545372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.546048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.547682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1082/2161] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:29 2022] DELETE /identity/v3/projects/8fecfc501af3473d94260aa9467ad9c4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:29.552068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:29.553683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:29.581559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:29.583412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1080/2162] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:29.593863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9509676-966b-420b-bcad-91d3d9adf627 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:29.595661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b9509676-966b-420b-bcad-91d3d9adf627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:29.629153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f3a4d-7c7f-4c50-914e-67767cbfb34a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:29.637738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:29.638623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:29.638904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:29.639179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] PATH_INFO: `/v3/users/aeb843ad7e9748e18ca9de0f5c45f5b6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:29.640412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb843ad7e9748e18ca9de0f5c45f5b6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:29.642643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:29.680839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9509676-966b-420b-bcad-91d3d9adf627 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:29.685265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34270cf9-3648-4de0-88b3-af9b5ab5d2fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:29.686211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270cf9-3648-4de0-88b3-af9b5ab5d2fd service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:29.686486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270cf9-3648-4de0-88b3-af9b5ab5d2fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:29.686768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270cf9-3648-4de0-88b3-af9b5ab5d2fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:29.698803 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34270cf9-3648-4de0-88b3-af9b5ab5d2fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:29.700434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34270cf9-3648-4de0-88b3-af9b5ab5d2fd service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:29.709384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb843ad7e9748e18ca9de0f5c45f5b6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.710893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1081/2163] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:35:29.721638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb843ad7e9748e18ca9de0f5c45f5b6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.722391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb843ad7e9748e18ca9de0f5c45f5b6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.735503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb843ad7e9748e18ca9de0f5c45f5b6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.744337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb843ad7e9748e18ca9de0f5c45f5b6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:29.753048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1083/2164] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:29 2022] DELETE /identity/v3/users/aeb843ad7e9748e18ca9de0f5c45f5b6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:35:30.063447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34270cf9-3648-4de0-88b3-af9b5ab5d2fd service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:30.065050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-34270cf9-3648-4de0-88b3-af9b5ab5d2fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:30.082091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34270cf9-3648-4de0-88b3-af9b5ab5d2fd service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:30.135332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:30.161550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:30.162421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.162703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.162974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] PATH_INFO: `/v3/projects/a61e8cc1224e4aec9a77997ce2aa8483` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.164152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a61e8cc1224e4aec9a77997ce2aa8483)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:30.166814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:30.194160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:30.195876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:30.219152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f84745-a5e4-42ec-adb3-13d2e4ab2512 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:30.235434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a61e8cc1224e4aec9a77997ce2aa8483', 'request_id': 'req-fdab6d20-eea2-4980-addf-90d0acb75f43'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:30.245620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a61e8cc1224e4aec9a77997ce2aa8483', 'request_id': 'req-fdab6d20-eea2-4980-addf-90d0acb75f43'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:30.252677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a61e8cc1224e4aec9a77997ce2aa8483', 'request_id': 'req-fdab6d20-eea2-4980-addf-90d0acb75f43'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:30.274374 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] The token cache is being invalidate because project a61e8cc1224e4aec9a77997ce2aa8483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:30.275237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:30.276383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fdab6d20-eea2-4980-addf-90d0acb75f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:30.277976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1082/2165] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:30 2022] DELETE /identity/v3/projects/a61e8cc1224e4aec9a77997ce2aa8483 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:30.296830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:30.297963 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.298319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.298654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] PATH_INFO: `/v3/projects/7dbe3ba631124c899cbdda93dd23c3e5` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.300126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dbe3ba631124c899cbdda93dd23c3e5)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:30.302739 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:30.357890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:30.361857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dbe3ba631124c899cbdda93dd23c3e5', 'request_id': 'req-23f18922-fe00-41d4-8091-bd8f25df45f8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:30.371792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dbe3ba631124c899cbdda93dd23c3e5', 'request_id': 'req-23f18922-fe00-41d4-8091-bd8f25df45f8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:30.381901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dbe3ba631124c899cbdda93dd23c3e5', 'request_id': 'req-23f18922-fe00-41d4-8091-bd8f25df45f8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:30.396969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] The token cache is being invalidate because project 7dbe3ba631124c899cbdda93dd23c3e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:30.397319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:30.398014 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-23f18922-fe00-41d4-8091-bd8f25df45f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:30.399680 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1084/2166] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:30 2022] DELETE /identity/v3/projects/7dbe3ba631124c899cbdda93dd23c3e5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:30.406888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a82686-f0c1-4429-893c-7fe9d614fe18 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.407424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a82686-f0c1-4429-893c-7fe9d614fe18 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.407424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a82686-f0c1-4429-893c-7fe9d614fe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.428986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a8a82686-f0c1-4429-893c-7fe9d614fe18 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:30.430206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a8a82686-f0c1-4429-893c-7fe9d614fe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:30.496119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1083/2167] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:30.504949 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d73a6-f090-4b44-b342-b53c47064df9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.505242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d73a6-f090-4b44-b342-b53c47064df9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.505531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d73a6-f090-4b44-b342-b53c47064df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.523747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-053d73a6-f090-4b44-b342-b53c47064df9 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:30.524960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-053d73a6-f090-4b44-b342-b53c47064df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:30.533663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1085/2168] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:30.537385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a82686-f0c1-4429-893c-7fe9d614fe18 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:30.547961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a82686-f0c1-4429-893c-7fe9d614fe18 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:30.552580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1907d-03de-4bdc-aa44-312c2078dd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:30.553237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1907d-03de-4bdc-aa44-312c2078dd2d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.553526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1907d-03de-4bdc-aa44-312c2078dd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.553808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1907d-03de-4bdc-aa44-312c2078dd2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.554544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1907d-03de-4bdc-aa44-312c2078dd2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:35:30.554859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1907d-03de-4bdc-aa44-312c2078dd2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:30.557303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1907d-03de-4bdc-aa44-312c2078dd2d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:30.571341 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1084/2169] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:35:30.574190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d73a6-f090-4b44-b342-b53c47064df9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:30.583622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d73a6-f090-4b44-b342-b53c47064df9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:30.588403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa3e15-64a3-40c1-8a97-66ea7b7011b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:30.589059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa3e15-64a3-40c1-8a97-66ea7b7011b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.589346 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa3e15-64a3-40c1-8a97-66ea7b7011b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.589625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa3e15-64a3-40c1-8a97-66ea7b7011b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.590319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa3e15-64a3-40c1-8a97-66ea7b7011b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:30.592261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa3e15-64a3-40c1-8a97-66ea7b7011b5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:30.618487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1086/2170] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:30 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:30.623269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1907d-03de-4bdc-aa44-312c2078dd2d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:30.634612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1907d-03de-4bdc-aa44-312c2078dd2d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:30.687579 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:30.702438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c40d4-fc58-4407-b8e0-5644c7f2dfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:30.703197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c40d4-fc58-4407-b8e0-5644c7f2dfc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.703479 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c40d4-fc58-4407-b8e0-5644c7f2dfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.703760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c40d4-fc58-4407-b8e0-5644c7f2dfc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.704518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c40d4-fc58-4407-b8e0-5644c7f2dfc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:30.706422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c40d4-fc58-4407-b8e0-5644c7f2dfc1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:30.731777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1085/2171] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:30 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:30.736114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa3e15-64a3-40c1-8a97-66ea7b7011b5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:30.748552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa3e15-64a3-40c1-8a97-66ea7b7011b5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:30.753333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315a218-0538-47e3-9ae9-46d6c70fc985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:30.754225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315a218-0538-47e3-9ae9-46d6c70fc985 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.754634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315a218-0538-47e3-9ae9-46d6c70fc985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.754915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315a218-0538-47e3-9ae9-46d6c70fc985 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.755648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315a218-0538-47e3-9ae9-46d6c70fc985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:30.757281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315a218-0538-47e3-9ae9-46d6c70fc985 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:30.767961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1087/2172] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:30.772681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c40d4-fc58-4407-b8e0-5644c7f2dfc1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:30.783353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c40d4-fc58-4407-b8e0-5644c7f2dfc1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:30.787848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabd0f3-6c3f-41ba-b755-94c0b68665fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:30.788539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabd0f3-6c3f-41ba-b755-94c0b68665fc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.788819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabd0f3-6c3f-41ba-b755-94c0b68665fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.789096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabd0f3-6c3f-41ba-b755-94c0b68665fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.789845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabd0f3-6c3f-41ba-b755-94c0b68665fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:30.792156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabd0f3-6c3f-41ba-b755-94c0b68665fc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:30.802537 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1086/2173] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:30.807322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315a218-0538-47e3-9ae9-46d6c70fc985 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:30.817317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315a218-0538-47e3-9ae9-46d6c70fc985 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:30.821939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca0d1f-288b-4c61-90e2-6fc9f929d979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:30.823206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca0d1f-288b-4c61-90e2-6fc9f929d979 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.823486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca0d1f-288b-4c61-90e2-6fc9f929d979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.823762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca0d1f-288b-4c61-90e2-6fc9f929d979 admin admin] PATH_INFO: `/v3/projects/864b427c4e914a5eb453789750f41ebd/users/2fb8a631d0c6415b94459b27a973c8f6/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.839244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca0d1f-288b-4c61-90e2-6fc9f929d979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=864b427c4e914a5eb453789750f41ebd, user_id=2fb8a631d0c6415b94459b27a973c8f6, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:30.842430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca0d1f-288b-4c61-90e2-6fc9f929d979 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:30.864962 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1088/2174] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:30 2022] PUT /identity/v3/projects/864b427c4e914a5eb453789750f41ebd/users/2fb8a631d0c6415b94459b27a973c8f6/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:35:30.877080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa4d0-fed2-4530-a919-9b96b33079e5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:30.877373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa4d0-fed2-4530-a919-9b96b33079e5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:30.877778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa4d0-fed2-4530-a919-9b96b33079e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:30.905612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8abaa4d0-fed2-4530-a919-9b96b33079e5 None None] MFA Rules not processed for user `2fb8a631d0c6415b94459b27a973c8f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:30.906819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8abaa4d0-fed2-4530-a919-9b96b33079e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:30.973075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1087/2175] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:30.978807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca0d1f-288b-4c61-90e2-6fc9f929d979 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:31.040924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca0d1f-288b-4c61-90e2-6fc9f929d979 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:31.045579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6261a-1046-4c0d-ab82-be2279431d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:31.046235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6261a-1046-4c0d-ab82-be2279431d7c admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:31.046562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6261a-1046-4c0d-ab82-be2279431d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:31.046840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6261a-1046-4c0d-ab82-be2279431d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:31.047693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6261a-1046-4c0d-ab82-be2279431d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:31.049636 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6261a-1046-4c0d-ab82-be2279431d7c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:31.082056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1089/2176] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:30 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:31.083379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa4d0-fed2-4530-a919-9b96b33079e5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:31.096642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa4d0-fed2-4530-a919-9b96b33079e5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:31.159286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717af9e-3f10-4861-9114-deda88404bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:31.159968 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717af9e-3f10-4861-9114-deda88404bef admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:31.160255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717af9e-3f10-4861-9114-deda88404bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:31.160567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717af9e-3f10-4861-9114-deda88404bef admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:31.161254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717af9e-3f10-4861-9114-deda88404bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:31.163169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717af9e-3f10-4861-9114-deda88404bef admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:31.184143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1088/2177] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:31 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:35:31.188289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6261a-1046-4c0d-ab82-be2279431d7c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:31.200553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6261a-1046-4c0d-ab82-be2279431d7c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:31.205325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad8c48-bb30-4911-ab57-57b5b43bad1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:31.206021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad8c48-bb30-4911-ab57-57b5b43bad1d admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:31.206307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad8c48-bb30-4911-ab57-57b5b43bad1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:31.206587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad8c48-bb30-4911-ab57-57b5b43bad1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:31.207329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad8c48-bb30-4911-ab57-57b5b43bad1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:31.208984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad8c48-bb30-4911-ab57-57b5b43bad1d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:31.220562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1090/2178] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:31.224759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717af9e-3f10-4861-9114-deda88404bef admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:31.234984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717af9e-3f10-4861-9114-deda88404bef admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:31.239636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78e4db-e16d-4c31-be67-d49fa4a8b1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:31.240542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78e4db-e16d-4c31-be67-d49fa4a8b1db admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:31.240828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78e4db-e16d-4c31-be67-d49fa4a8b1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:31.241107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78e4db-e16d-4c31-be67-d49fa4a8b1db admin admin] PATH_INFO: `/v3/projects/dfc8dca03d1840f19f351be6f59d63f8/users/2657f9562e7b49fb80f97dd958b5c9b0/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:31.256848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78e4db-e16d-4c31-be67-d49fa4a8b1db admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc8dca03d1840f19f351be6f59d63f8, user_id=2657f9562e7b49fb80f97dd958b5c9b0, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:31.260100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78e4db-e16d-4c31-be67-d49fa4a8b1db admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:31.273817 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1089/2179] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:31 2022] PUT /identity/v3/projects/dfc8dca03d1840f19f351be6f59d63f8/users/2657f9562e7b49fb80f97dd958b5c9b0/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:31.278226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad8c48-bb30-4911-ab57-57b5b43bad1d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:31.288160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad8c48-bb30-4911-ab57-57b5b43bad1d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:31.336812 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:31.355552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec0b70-afb9-49ff-b25c-18170d7b0605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:31.356230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0b70-afb9-49ff-b25c-18170d7b0605 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:31.356568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0b70-afb9-49ff-b25c-18170d7b0605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:31.356848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0b70-afb9-49ff-b25c-18170d7b0605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:31.357611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec0b70-afb9-49ff-b25c-18170d7b0605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:31.359311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec0b70-afb9-49ff-b25c-18170d7b0605 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:31.375743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1091/2180] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:31.377091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78e4db-e16d-4c31-be67-d49fa4a8b1db admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:31.388759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78e4db-e16d-4c31-be67-d49fa4a8b1db admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:31.393665 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a9030-3ec2-4ac1-b0db-bc7094776692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:31.394632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a9030-3ec2-4ac1-b0db-bc7094776692 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:31.394991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a9030-3ec2-4ac1-b0db-bc7094776692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:31.395339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a9030-3ec2-4ac1-b0db-bc7094776692 admin admin] PATH_INFO: `/v3/projects/dfc8dca03d1840f19f351be6f59d63f8/users/2657f9562e7b49fb80f97dd958b5c9b0/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:31.397460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a9030-3ec2-4ac1-b0db-bc7094776692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc8dca03d1840f19f351be6f59d63f8, user_id=2657f9562e7b49fb80f97dd958b5c9b0, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:31.400793 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a9030-3ec2-4ac1-b0db-bc7094776692 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:31.414481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1090/2181] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:31 2022] PUT /identity/v3/projects/dfc8dca03d1840f19f351be6f59d63f8/users/2657f9562e7b49fb80f97dd958b5c9b0/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:31.425060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbafd7-1dd5-4e05-9a63-ee8848889ff2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:31.425356 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbafd7-1dd5-4e05-9a63-ee8848889ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:31.425640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbafd7-1dd5-4e05-9a63-ee8848889ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:31.452430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2edbafd7-1dd5-4e05-9a63-ee8848889ff2 None None] MFA Rules not processed for user `2657f9562e7b49fb80f97dd958b5c9b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:31.453632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2edbafd7-1dd5-4e05-9a63-ee8848889ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:31.525821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1092/2182] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:31.536715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a9030-3ec2-4ac1-b0db-bc7094776692 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:31.538440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-601a9030-3ec2-4ac1-b0db-bc7094776692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:31.560337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:31.598694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a9030-3ec2-4ac1-b0db-bc7094776692 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:31.602201 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b864f2-74d8-4f15-b76a-3a0819671631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:31.602946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b864f2-74d8-4f15-b76a-3a0819671631 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:31.603233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b864f2-74d8-4f15-b76a-3a0819671631 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:31.603511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b864f2-74d8-4f15-b76a-3a0819671631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:31.612275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b864f2-74d8-4f15-b76a-3a0819671631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:31.613736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b864f2-74d8-4f15-b76a-3a0819671631 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:31.624181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1091/2183] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:31.754611 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbafd7-1dd5-4e05-9a63-ee8848889ff2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:31.819521 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbafd7-1dd5-4e05-9a63-ee8848889ff2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:31.824877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:31.825746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:31.826026 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:31.826301 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] PATH_INFO: `/v3/users/2fb8a631d0c6415b94459b27a973c8f6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:31.827463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb8a631d0c6415b94459b27a973c8f6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:31.829426 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:31.876529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb8a631d0c6415b94459b27a973c8f6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:31.883800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb8a631d0c6415b94459b27a973c8f6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:31.885341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb8a631d0c6415b94459b27a973c8f6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:31.894251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:31.899934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb8a631d0c6415b94459b27a973c8f6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:31.907596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb8a631d0c6415b94459b27a973c8f6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:31.913984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1093/2184] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:31 2022] DELETE /identity/v3/users/2fb8a631d0c6415b94459b27a973c8f6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:31.919263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5904285-f1e3-4320-9f6b-9f7a73b75368 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:31.919640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5904285-f1e3-4320-9f6b-9f7a73b75368 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:31.919937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5904285-f1e3-4320-9f6b-9f7a73b75368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:31.938479 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f5904285-f1e3-4320-9f6b-9f7a73b75368 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:31.939735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f5904285-f1e3-4320-9f6b-9f7a73b75368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:32.002007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1092/2185] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:32.012349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:32.013765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:32.074902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33012c55-5394-42a1-b9e1-5cd4ba9ea84a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:32.078849 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd129d36-a182-4a99-8552-8eed37cca7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:32.079498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd129d36-a182-4a99-8552-8eed37cca7dd service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:32.079782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd129d36-a182-4a99-8552-8eed37cca7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:32.080061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd129d36-a182-4a99-8552-8eed37cca7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:32.089977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd129d36-a182-4a99-8552-8eed37cca7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:32.091449 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd129d36-a182-4a99-8552-8eed37cca7dd service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:32.100768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1094/2186] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:32.380949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5904285-f1e3-4320-9f6b-9f7a73b75368 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:32.382352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f5904285-f1e3-4320-9f6b-9f7a73b75368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:32.395533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5904285-f1e3-4320-9f6b-9f7a73b75368 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:32.400112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:32.401018 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:32.401297 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:32.401570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] PATH_INFO: `/v3/projects/864b427c4e914a5eb453789750f41ebd` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:32.402751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=864b427c4e914a5eb453789750f41ebd)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:32.404703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:32.453964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '864b427c4e914a5eb453789750f41ebd', 'request_id': 'req-7f5de7d5-028e-4a73-8b13-5b5f56901444'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:32.462625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '864b427c4e914a5eb453789750f41ebd', 'request_id': 'req-7f5de7d5-028e-4a73-8b13-5b5f56901444'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:32.468946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '864b427c4e914a5eb453789750f41ebd', 'request_id': 'req-7f5de7d5-028e-4a73-8b13-5b5f56901444'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:32.480631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] The token cache is being invalidate because project 864b427c4e914a5eb453789750f41ebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:32.480921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:32.481652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:32.483834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1093/2187] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:32 2022] DELETE /identity/v3/projects/864b427c4e914a5eb453789750f41ebd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:32.487373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd129d36-a182-4a99-8552-8eed37cca7dd service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:32.498192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd129d36-a182-4a99-8552-8eed37cca7dd service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:32.558343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:32.559218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:32.559503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:32.559787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] PATH_INFO: `/v3/users/2657f9562e7b49fb80f97dd958b5c9b0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:32.561005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2657f9562e7b49fb80f97dd958b5c9b0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:32.562937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:32.613887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2657f9562e7b49fb80f97dd958b5c9b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:32.622043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2657f9562e7b49fb80f97dd958b5c9b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:32.622726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2657f9562e7b49fb80f97dd958b5c9b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:32.635553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2657f9562e7b49fb80f97dd958b5c9b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:32.642997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f0147a53-cf01-4192-9d0a-f520f515eb10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2657f9562e7b49fb80f97dd958b5c9b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:32.649690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1095/2188] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:32 2022] DELETE /identity/v3/users/2657f9562e7b49fb80f97dd958b5c9b0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:32.944724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:32.946232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:32.959514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5de7d5-028e-4a73-8b13-5b5f56901444 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.005650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:33.022791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.023736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.024006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.024277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] PATH_INFO: `/v3/projects/dfc8dca03d1840f19f351be6f59d63f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.026264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc8dca03d1840f19f351be6f59d63f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.028392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.075705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc8dca03d1840f19f351be6f59d63f8', 'request_id': 'req-fe6978fb-8219-464c-a1ea-89e65af6c6ee'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:33.083780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc8dca03d1840f19f351be6f59d63f8', 'request_id': 'req-fe6978fb-8219-464c-a1ea-89e65af6c6ee'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:33.091912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc8dca03d1840f19f351be6f59d63f8', 'request_id': 'req-fe6978fb-8219-464c-a1ea-89e65af6c6ee'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:33.104039 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] The token cache is being invalidate because project dfc8dca03d1840f19f351be6f59d63f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:33.104356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:33.105028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe6978fb-8219-464c-a1ea-89e65af6c6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:33.106575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1094/2189] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:32 2022] DELETE /identity/v3/projects/dfc8dca03d1840f19f351be6f59d63f8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:33.111476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a094b70-b35d-4244-93f7-9622b1acd145 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.111843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a094b70-b35d-4244-93f7-9622b1acd145 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.112204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a094b70-b35d-4244-93f7-9622b1acd145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.124433 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:33.131046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8a094b70-b35d-4244-93f7-9622b1acd145 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:33.132640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8a094b70-b35d-4244-93f7-9622b1acd145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:33.194726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1096/2190] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:33.204225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8495648-632b-4c1a-840c-81c8c74981b5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.204699 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8495648-632b-4c1a-840c-81c8c74981b5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.205153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8495648-632b-4c1a-840c-81c8c74981b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.224318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d8495648-632b-4c1a-840c-81c8c74981b5 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:33.225549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d8495648-632b-4c1a-840c-81c8c74981b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:33.233778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1095/2191] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:33.238098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a094b70-b35d-4244-93f7-9622b1acd145 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.248508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a094b70-b35d-4244-93f7-9622b1acd145 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.254249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6893d-2b4f-47a4-8596-85361d2f507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.254918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6893d-2b4f-47a4-8596-85361d2f507a admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.255206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6893d-2b4f-47a4-8596-85361d2f507a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.255489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6893d-2b4f-47a4-8596-85361d2f507a admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.256245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6893d-2b4f-47a4-8596-85361d2f507a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:35:33.256593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6893d-2b4f-47a4-8596-85361d2f507a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.258214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6893d-2b4f-47a4-8596-85361d2f507a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.272001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1097/2192] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:35:33.276377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8495648-632b-4c1a-840c-81c8c74981b5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.285530 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8495648-632b-4c1a-840c-81c8c74981b5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.290232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ab060-6dfb-449a-a293-021e2f4d4ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.291017 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ab060-6dfb-449a-a293-021e2f4d4ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.291304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ab060-6dfb-449a-a293-021e2f4d4ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.291580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ab060-6dfb-449a-a293-021e2f4d4ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.292280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ab060-6dfb-449a-a293-021e2f4d4ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.294212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ab060-6dfb-449a-a293-021e2f4d4ee4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.323943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1096/2193] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:33 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:33.326118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6893d-2b4f-47a4-8596-85361d2f507a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.337280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6893d-2b4f-47a4-8596-85361d2f507a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.395359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18008a-5411-4f80-b032-b15d6c13b4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.396049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18008a-5411-4f80-b032-b15d6c13b4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.396360 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18008a-5411-4f80-b032-b15d6c13b4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.396642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18008a-5411-4f80-b032-b15d6c13b4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.397357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18008a-5411-4f80-b032-b15d6c13b4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.399300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18008a-5411-4f80-b032-b15d6c13b4c6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.419955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1098/2194] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:33 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:33.424168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ab060-6dfb-449a-a293-021e2f4d4ee4 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.435285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ab060-6dfb-449a-a293-021e2f4d4ee4 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.440094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb1368-760b-4e30-81fc-cdd5a45e1719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.440836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1368-760b-4e30-81fc-cdd5a45e1719 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.441117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1368-760b-4e30-81fc-cdd5a45e1719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.441391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1368-760b-4e30-81fc-cdd5a45e1719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.442239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb1368-760b-4e30-81fc-cdd5a45e1719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.443922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb1368-760b-4e30-81fc-cdd5a45e1719 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.455582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1097/2195] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:33.459621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18008a-5411-4f80-b032-b15d6c13b4c6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.469768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18008a-5411-4f80-b032-b15d6c13b4c6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.474500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cf6d2-9230-4036-9240-117e7747dfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.475174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cf6d2-9230-4036-9240-117e7747dfa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.475461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cf6d2-9230-4036-9240-117e7747dfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.475742 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cf6d2-9230-4036-9240-117e7747dfa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.476512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cf6d2-9230-4036-9240-117e7747dfa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.478745 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cf6d2-9230-4036-9240-117e7747dfa5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.494002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1099/2196] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:33.498135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb1368-760b-4e30-81fc-cdd5a45e1719 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.508189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb1368-760b-4e30-81fc-cdd5a45e1719 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.513516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528fe66-0e99-4dc4-b4fe-015175c7cb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.514428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528fe66-0e99-4dc4-b4fe-015175c7cb6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.514714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528fe66-0e99-4dc4-b4fe-015175c7cb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.515013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528fe66-0e99-4dc4-b4fe-015175c7cb6b admin admin] PATH_INFO: `/v3/projects/d781120794e74740ad47da35b805cae7/users/b261c2cb55d5454bb2e0dfda09762e93/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.530466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528fe66-0e99-4dc4-b4fe-015175c7cb6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d781120794e74740ad47da35b805cae7, user_id=b261c2cb55d5454bb2e0dfda09762e93, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.533166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528fe66-0e99-4dc4-b4fe-015175c7cb6b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.547936 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1098/2197] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:33 2022] PUT /identity/v3/projects/d781120794e74740ad47da35b805cae7/users/b261c2cb55d5454bb2e0dfda09762e93/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:33.559745 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3fb00-649f-4fbf-9207-bf69c5463a5b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.560031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3fb00-649f-4fbf-9207-bf69c5463a5b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.560333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3fb00-649f-4fbf-9207-bf69c5463a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.588481 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3fa3fb00-649f-4fbf-9207-bf69c5463a5b None None] MFA Rules not processed for user `b261c2cb55d5454bb2e0dfda09762e93`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:33.589673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3fa3fb00-649f-4fbf-9207-bf69c5463a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:33.657076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1100/2198] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:33.661484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528fe66-0e99-4dc4-b4fe-015175c7cb6b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.730367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528fe66-0e99-4dc4-b4fe-015175c7cb6b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.735044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642811f-d062-462a-81a0-823d566eaa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.735822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642811f-d062-462a-81a0-823d566eaa41 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.736110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642811f-d062-462a-81a0-823d566eaa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.736420 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642811f-d062-462a-81a0-823d566eaa41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.737119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642811f-d062-462a-81a0-823d566eaa41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.739051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642811f-d062-462a-81a0-823d566eaa41 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.754043 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:33.768378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1099/2199] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:33 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:33.773084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3fb00-649f-4fbf-9207-bf69c5463a5b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.779000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:33.785215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3fb00-649f-4fbf-9207-bf69c5463a5b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.849805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b138b2c-1f47-4cc5-bc75-15c06321933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.850495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b138b2c-1f47-4cc5-bc75-15c06321933a admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.850819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b138b2c-1f47-4cc5-bc75-15c06321933a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.851200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b138b2c-1f47-4cc5-bc75-15c06321933a admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.852001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b138b2c-1f47-4cc5-bc75-15c06321933a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.854219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b138b2c-1f47-4cc5-bc75-15c06321933a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.875697 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1101/2200] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:33 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 17:35:33.879847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642811f-d062-462a-81a0-823d566eaa41 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.891696 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642811f-d062-462a-81a0-823d566eaa41 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.896881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee79b-3463-4ad7-b2c4-ce6e40abac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.897574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee79b-3463-4ad7-b2c4-ce6e40abac87 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.897868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee79b-3463-4ad7-b2c4-ce6e40abac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.898148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee79b-3463-4ad7-b2c4-ce6e40abac87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.898897 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ee79b-3463-4ad7-b2c4-ce6e40abac87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.900871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ee79b-3463-4ad7-b2c4-ce6e40abac87 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.912328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1100/2201] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:33.917367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b138b2c-1f47-4cc5-bc75-15c06321933a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.927824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b138b2c-1f47-4cc5-bc75-15c06321933a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:33.933165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee87cf6-d3bb-421f-b6f4-e307b54dec9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:33.934044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee87cf6-d3bb-421f-b6f4-e307b54dec9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:33.934435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee87cf6-d3bb-421f-b6f4-e307b54dec9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:33.934714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee87cf6-d3bb-421f-b6f4-e307b54dec9d admin admin] PATH_INFO: `/v3/projects/b6a119b5cc3f49d49537dc6cb3b618a1/users/793e6fa3b12846188bdebc94dcac5e5e/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:33.950618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee87cf6-d3bb-421f-b6f4-e307b54dec9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a119b5cc3f49d49537dc6cb3b618a1, user_id=793e6fa3b12846188bdebc94dcac5e5e, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:33.953494 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee87cf6-d3bb-421f-b6f4-e307b54dec9d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:33.967428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1102/2202] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:33 2022] PUT /identity/v3/projects/b6a119b5cc3f49d49537dc6cb3b618a1/users/793e6fa3b12846188bdebc94dcac5e5e/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:33.971449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee79b-3463-4ad7-b2c4-ce6e40abac87 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:33.981943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee79b-3463-4ad7-b2c4-ce6e40abac87 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:34.039467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d00ff5-6720-4031-9138-8b5a4f444dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:34.040221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d00ff5-6720-4031-9138-8b5a4f444dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:34.040615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d00ff5-6720-4031-9138-8b5a4f444dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:34.040992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d00ff5-6720-4031-9138-8b5a4f444dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:34.041742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d00ff5-6720-4031-9138-8b5a4f444dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:34.043394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d00ff5-6720-4031-9138-8b5a4f444dc3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:34.053953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1101/2203] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:34.058726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee87cf6-d3bb-421f-b6f4-e307b54dec9d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:34.069382 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee87cf6-d3bb-421f-b6f4-e307b54dec9d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:34.075004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369b4d0-6d6f-4525-b65f-830eaa092fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:34.075927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369b4d0-6d6f-4525-b65f-830eaa092fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:34.076213 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369b4d0-6d6f-4525-b65f-830eaa092fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:34.076539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369b4d0-6d6f-4525-b65f-830eaa092fd5 admin admin] PATH_INFO: `/v3/projects/b6a119b5cc3f49d49537dc6cb3b618a1/users/793e6fa3b12846188bdebc94dcac5e5e/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:34.078525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369b4d0-6d6f-4525-b65f-830eaa092fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a119b5cc3f49d49537dc6cb3b618a1, user_id=793e6fa3b12846188bdebc94dcac5e5e, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:34.081257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369b4d0-6d6f-4525-b65f-830eaa092fd5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:34.092154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1103/2204] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:34 2022] PUT /identity/v3/projects/b6a119b5cc3f49d49537dc6cb3b618a1/users/793e6fa3b12846188bdebc94dcac5e5e/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:35:34.102519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b5a10-1632-45b6-ba67-b9c990dc7e10 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:34.102892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b5a10-1632-45b6-ba67-b9c990dc7e10 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:34.103137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b5a10-1632-45b6-ba67-b9c990dc7e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:34.131383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fd4b5a10-1632-45b6-ba67-b9c990dc7e10 None None] MFA Rules not processed for user `793e6fa3b12846188bdebc94dcac5e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:34.132619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fd4b5a10-1632-45b6-ba67-b9c990dc7e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:34.202382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1102/2205] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:34.214202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369b4d0-6d6f-4525-b65f-830eaa092fd5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:34.215919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0369b4d0-6d6f-4525-b65f-830eaa092fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:34.283523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369b4d0-6d6f-4525-b65f-830eaa092fd5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:34.287563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c24105-e36d-4570-9dc8-b519c7546ceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:34.288220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c24105-e36d-4570-9dc8-b519c7546ceb service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:34.288555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c24105-e36d-4570-9dc8-b519c7546ceb service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:34.288810 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c24105-e36d-4570-9dc8-b519c7546ceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:34.299478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c24105-e36d-4570-9dc8-b519c7546ceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:34.300983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c24105-e36d-4570-9dc8-b519c7546ceb service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:34.311398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1104/2206] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:35:34.900604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b5a10-1632-45b6-ba67-b9c990dc7e10 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:34.991086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b5a10-1632-45b6-ba67-b9c990dc7e10 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:34.996026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:34.996949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:34.997231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:34.997503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] PATH_INFO: `/v3/users/b261c2cb55d5454bb2e0dfda09762e93` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:34.998711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b261c2cb55d5454bb2e0dfda09762e93)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:35.000723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:35.006224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:35.048691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b261c2cb55d5454bb2e0dfda09762e93'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.049354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b261c2cb55d5454bb2e0dfda09762e93'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.049960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b261c2cb55d5454bb2e0dfda09762e93'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.057408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b261c2cb55d5454bb2e0dfda09762e93'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.076693 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b261c2cb55d5454bb2e0dfda09762e93'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.089864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1103/2207] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:34 2022] DELETE /identity/v3/users/b261c2cb55d5454bb2e0dfda09762e93 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:35.094842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffa8f4-b8f8-400b-946a-fb054cdebf9b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:35.095354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffa8f4-b8f8-400b-946a-fb054cdebf9b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:35.095456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffa8f4-b8f8-400b-946a-fb054cdebf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:35.113965 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e2ffa8f4-b8f8-400b-946a-fb054cdebf9b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:35.115156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e2ffa8f4-b8f8-400b-946a-fb054cdebf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:35.167997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:35.178545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1105/2208] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:35:35.188884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:35.190276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:35.255988 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a7854-fd4a-4c4e-bb90-9e8b906429be admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:35.259533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25642298-3190-4630-a65e-2089b7d21bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:35.260189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25642298-3190-4630-a65e-2089b7d21bdd service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:35.260528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25642298-3190-4630-a65e-2089b7d21bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:35.260800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25642298-3190-4630-a65e-2089b7d21bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:35.270045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25642298-3190-4630-a65e-2089b7d21bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:35.271468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25642298-3190-4630-a65e-2089b7d21bdd service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:35.282108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1104/2209] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:35.549602 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffa8f4-b8f8-400b-946a-fb054cdebf9b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:35.551033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e2ffa8f4-b8f8-400b-946a-fb054cdebf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:35.563784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffa8f4-b8f8-400b-946a-fb054cdebf9b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:35.568567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:35.569515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:35.569799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:35.570142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] PATH_INFO: `/v3/projects/d781120794e74740ad47da35b805cae7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:35.571858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d781120794e74740ad47da35b805cae7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:35.574280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:35.624504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd781120794e74740ad47da35b805cae7', 'request_id': 'req-ee15a5aa-828c-423f-abb4-b48d60041f94'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.630454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd781120794e74740ad47da35b805cae7', 'request_id': 'req-ee15a5aa-828c-423f-abb4-b48d60041f94'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.638718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd781120794e74740ad47da35b805cae7', 'request_id': 'req-ee15a5aa-828c-423f-abb4-b48d60041f94'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.651601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] The token cache is being invalidate because project d781120794e74740ad47da35b805cae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:35.651889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.652558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.654081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1106/2210] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:35 2022] DELETE /identity/v3/projects/d781120794e74740ad47da35b805cae7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:35.658172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25642298-3190-4630-a65e-2089b7d21bdd service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:35.667814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25642298-3190-4630-a65e-2089b7d21bdd service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:35.725929 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:35.726811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:35.727099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:35.727382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] PATH_INFO: `/v3/users/793e6fa3b12846188bdebc94dcac5e5e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:35.728811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=793e6fa3b12846188bdebc94dcac5e5e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:35.730768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:35.765432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:35.780909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793e6fa3b12846188bdebc94dcac5e5e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.781679 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793e6fa3b12846188bdebc94dcac5e5e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.782406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793e6fa3b12846188bdebc94dcac5e5e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.789723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793e6fa3b12846188bdebc94dcac5e5e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.802811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3b5873c9-633d-4bba-ba04-85a6e2f5cd0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793e6fa3b12846188bdebc94dcac5e5e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:35.816306 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1105/2211] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:35 2022] DELETE /identity/v3/users/793e6fa3b12846188bdebc94dcac5e5e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:36.108763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:36.110278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:36.124648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15a5aa-828c-423f-abb4-b48d60041f94 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:36.192940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:36.193805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.194109 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.194365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] PATH_INFO: `/v3/projects/b6a119b5cc3f49d49537dc6cb3b618a1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.195566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a119b5cc3f49d49537dc6cb3b618a1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:36.198719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:36.248472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a119b5cc3f49d49537dc6cb3b618a1', 'request_id': 'req-4007c7de-d939-489e-8129-566e6e81fdb5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:36.264375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a119b5cc3f49d49537dc6cb3b618a1', 'request_id': 'req-4007c7de-d939-489e-8129-566e6e81fdb5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:36.272021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a119b5cc3f49d49537dc6cb3b618a1', 'request_id': 'req-4007c7de-d939-489e-8129-566e6e81fdb5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:36.284261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] The token cache is being invalidate because project b6a119b5cc3f49d49537dc6cb3b618a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:36.284582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:36.285232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4007c7de-d939-489e-8129-566e6e81fdb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:36.286763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1107/2212] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:36 2022] DELETE /identity/v3/projects/b6a119b5cc3f49d49537dc6cb3b618a1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:36.292487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fba8e4-9d59-4799-aa92-4cb69ba71bee None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.292938 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fba8e4-9d59-4799-aa92-4cb69ba71bee None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.293059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fba8e4-9d59-4799-aa92-4cb69ba71bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.311027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-05fba8e4-9d59-4799-aa92-4cb69ba71bee None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:36.312321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-05fba8e4-9d59-4799-aa92-4cb69ba71bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:36.378963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1106/2213] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 17:35:36.387101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bae88a-e0a2-4fc6-bbd3-931af197809b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.387395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bae88a-e0a2-4fc6-bbd3-931af197809b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.387679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bae88a-e0a2-4fc6-bbd3-931af197809b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.406234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-32bae88a-e0a2-4fc6-bbd3-931af197809b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:36.407457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-32bae88a-e0a2-4fc6-bbd3-931af197809b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:36.415858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1108/2214] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:36.419907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fba8e4-9d59-4799-aa92-4cb69ba71bee None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:36.429427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fba8e4-9d59-4799-aa92-4cb69ba71bee None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:36.434852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cdaae-e30b-4308-a4d5-34f7b1eaefda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:36.435502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cdaae-e30b-4308-a4d5-34f7b1eaefda admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.435786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cdaae-e30b-4308-a4d5-34f7b1eaefda admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.436062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cdaae-e30b-4308-a4d5-34f7b1eaefda admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.436825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cdaae-e30b-4308-a4d5-34f7b1eaefda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:35:36.437132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cdaae-e30b-4308-a4d5-34f7b1eaefda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:36.438712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cdaae-e30b-4308-a4d5-34f7b1eaefda admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:36.451166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1107/2215] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:35:36.455650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bae88a-e0a2-4fc6-bbd3-931af197809b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:36.461324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:36.465337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bae88a-e0a2-4fc6-bbd3-931af197809b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:36.472002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6b610-22a2-4579-a4ee-facc72d1a2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:36.472816 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6b610-22a2-4579-a4ee-facc72d1a2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.473257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6b610-22a2-4579-a4ee-facc72d1a2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.473633 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6b610-22a2-4579-a4ee-facc72d1a2b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.476813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6b610-22a2-4579-a4ee-facc72d1a2b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:36.479974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6b610-22a2-4579-a4ee-facc72d1a2b1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:36.507080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1109/2216] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:36 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:36.511393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cdaae-e30b-4308-a4d5-34f7b1eaefda admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:36.521975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cdaae-e30b-4308-a4d5-34f7b1eaefda admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:36.582262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1708b21-ad76-4063-a58f-0ebf57ba2219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:36.582930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1708b21-ad76-4063-a58f-0ebf57ba2219 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.583211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1708b21-ad76-4063-a58f-0ebf57ba2219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.583483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1708b21-ad76-4063-a58f-0ebf57ba2219 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.584161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1708b21-ad76-4063-a58f-0ebf57ba2219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:36.586096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1708b21-ad76-4063-a58f-0ebf57ba2219 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:36.607278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1108/2217] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:36 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:36.610853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6b610-22a2-4579-a4ee-facc72d1a2b1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:36.621903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6b610-22a2-4579-a4ee-facc72d1a2b1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:36.626467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed53c5-e76e-4a13-92f0-4ffb4342902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:36.627115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed53c5-e76e-4a13-92f0-4ffb4342902b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.627395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed53c5-e76e-4a13-92f0-4ffb4342902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.627667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed53c5-e76e-4a13-92f0-4ffb4342902b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.628553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed53c5-e76e-4a13-92f0-4ffb4342902b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:36.630143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed53c5-e76e-4a13-92f0-4ffb4342902b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:36.640608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1110/2218] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:36.644600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1708b21-ad76-4063-a58f-0ebf57ba2219 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:36.655603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1708b21-ad76-4063-a58f-0ebf57ba2219 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:36.660439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9375ef3-b158-4558-83c5-06563a05babb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:36.661130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9375ef3-b158-4558-83c5-06563a05babb admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.661415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9375ef3-b158-4558-83c5-06563a05babb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.661689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9375ef3-b158-4558-83c5-06563a05babb admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.662444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9375ef3-b158-4558-83c5-06563a05babb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:36.664086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9375ef3-b158-4558-83c5-06563a05babb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:36.674943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1109/2219] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:36.679117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed53c5-e76e-4a13-92f0-4ffb4342902b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:36.690922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed53c5-e76e-4a13-92f0-4ffb4342902b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:36.695808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529483cc-1e23-4533-b247-9742a6902c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:36.696737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529483cc-1e23-4533-b247-9742a6902c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.697014 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529483cc-1e23-4533-b247-9742a6902c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.697550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529483cc-1e23-4533-b247-9742a6902c65 admin admin] PATH_INFO: `/v3/projects/fa00c52f6f714422be533a39ced08ef4/users/986bd53c012c49ca925577b3479706e5/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.712262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529483cc-1e23-4533-b247-9742a6902c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa00c52f6f714422be533a39ced08ef4, user_id=986bd53c012c49ca925577b3479706e5, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:36.714959 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529483cc-1e23-4533-b247-9742a6902c65 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:36.726924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1111/2220] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:36 2022] PUT /identity/v3/projects/fa00c52f6f714422be533a39ced08ef4/users/986bd53c012c49ca925577b3479706e5/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:36.738679 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e47cd0-786e-40c3-8fdd-0295309a758e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.738969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e47cd0-786e-40c3-8fdd-0295309a758e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.739267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e47cd0-786e-40c3-8fdd-0295309a758e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.764743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:36.768256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-67e47cd0-786e-40c3-8fdd-0295309a758e None None] MFA Rules not processed for user `986bd53c012c49ca925577b3479706e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:36.769952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-67e47cd0-786e-40c3-8fdd-0295309a758e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:36.838069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1110/2221] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:36.842591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529483cc-1e23-4533-b247-9742a6902c65 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:36.906920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529483cc-1e23-4533-b247-9742a6902c65 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:36.911657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80114024-a7d4-4c42-8281-c027ce4704ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:36.912342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80114024-a7d4-4c42-8281-c027ce4704ac admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:36.912620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80114024-a7d4-4c42-8281-c027ce4704ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:36.912900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80114024-a7d4-4c42-8281-c027ce4704ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:36.913595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80114024-a7d4-4c42-8281-c027ce4704ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:36.916594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80114024-a7d4-4c42-8281-c027ce4704ac admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:36.944357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1112/2222] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:36 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:36.949390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e47cd0-786e-40c3-8fdd-0295309a758e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:36.960627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e47cd0-786e-40c3-8fdd-0295309a758e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:37.016920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3402d7-8ad4-4c48-9a02-9b289802a9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:37.017898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3402d7-8ad4-4c48-9a02-9b289802a9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:37.018183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3402d7-8ad4-4c48-9a02-9b289802a9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:37.018452 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3402d7-8ad4-4c48-9a02-9b289802a9f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:37.019146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3402d7-8ad4-4c48-9a02-9b289802a9f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:37.021652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3402d7-8ad4-4c48-9a02-9b289802a9f7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:37.047769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1111/2223] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:36 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:37.049215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80114024-a7d4-4c42-8281-c027ce4704ac admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:37.060264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80114024-a7d4-4c42-8281-c027ce4704ac admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:37.065154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d35b9-21c2-4ee0-953e-8d7355db3bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:37.065854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d35b9-21c2-4ee0-953e-8d7355db3bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:37.066145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d35b9-21c2-4ee0-953e-8d7355db3bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:37.066425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d35b9-21c2-4ee0-953e-8d7355db3bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:37.067187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d35b9-21c2-4ee0-953e-8d7355db3bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:37.068834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d35b9-21c2-4ee0-953e-8d7355db3bc5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:37.079620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1113/2224] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:37.083879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3402d7-8ad4-4c48-9a02-9b289802a9f7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:37.094634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3402d7-8ad4-4c48-9a02-9b289802a9f7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:37.099251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7293cbe-2d7a-4b93-9238-dd411b01c4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:37.100401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7293cbe-2d7a-4b93-9238-dd411b01c4bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:37.100692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7293cbe-2d7a-4b93-9238-dd411b01c4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:37.100995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7293cbe-2d7a-4b93-9238-dd411b01c4bf admin admin] PATH_INFO: `/v3/projects/754d91e615744f2b8eaa07338c3b0d15/users/5e3e0e01cb9b4a9293e43f4c86c2ba78/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:37.116212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7293cbe-2d7a-4b93-9238-dd411b01c4bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=754d91e615744f2b8eaa07338c3b0d15, user_id=5e3e0e01cb9b4a9293e43f4c86c2ba78, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:37.119187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7293cbe-2d7a-4b93-9238-dd411b01c4bf admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:37.131829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1112/2225] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:37 2022] PUT /identity/v3/projects/754d91e615744f2b8eaa07338c3b0d15/users/5e3e0e01cb9b4a9293e43f4c86c2ba78/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:37.136258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d35b9-21c2-4ee0-953e-8d7355db3bc5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:37.146072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d35b9-21c2-4ee0-953e-8d7355db3bc5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:37.202081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14be7c-7e37-4dfa-8474-4b3909ff7ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:37.202785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14be7c-7e37-4dfa-8474-4b3909ff7ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:37.203050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14be7c-7e37-4dfa-8474-4b3909ff7ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:37.203354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14be7c-7e37-4dfa-8474-4b3909ff7ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:37.204075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14be7c-7e37-4dfa-8474-4b3909ff7ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:37.205676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14be7c-7e37-4dfa-8474-4b3909ff7ee7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:37.220202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1114/2226] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:37.221520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7293cbe-2d7a-4b93-9238-dd411b01c4bf admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:37.232259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7293cbe-2d7a-4b93-9238-dd411b01c4bf admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:37.237370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad5a25-207c-4309-9b7f-483feadfdc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:37.238268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad5a25-207c-4309-9b7f-483feadfdc30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:37.238554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad5a25-207c-4309-9b7f-483feadfdc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:37.238826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad5a25-207c-4309-9b7f-483feadfdc30 admin admin] PATH_INFO: `/v3/projects/754d91e615744f2b8eaa07338c3b0d15/users/5e3e0e01cb9b4a9293e43f4c86c2ba78/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:37.241107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad5a25-207c-4309-9b7f-483feadfdc30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=754d91e615744f2b8eaa07338c3b0d15, user_id=5e3e0e01cb9b4a9293e43f4c86c2ba78, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:37.243964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad5a25-207c-4309-9b7f-483feadfdc30 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:37.254529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1113/2227] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:37 2022] PUT /identity/v3/projects/754d91e615744f2b8eaa07338c3b0d15/users/5e3e0e01cb9b4a9293e43f4c86c2ba78/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:37.265116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3a85e-1a43-40c8-957d-163c8f92e9b7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:37.265408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3a85e-1a43-40c8-957d-163c8f92e9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:37.265688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3a85e-1a43-40c8-957d-163c8f92e9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:37.294155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7aa3a85e-1a43-40c8-957d-163c8f92e9b7 None None] MFA Rules not processed for user `5e3e0e01cb9b4a9293e43f4c86c2ba78`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:37.295467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7aa3a85e-1a43-40c8-957d-163c8f92e9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:37.300810 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:37.367382 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1115/2228] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:37.378101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad5a25-207c-4309-9b7f-483feadfdc30 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:37.379843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-06ad5a25-207c-4309-9b7f-483feadfdc30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:37.449980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad5a25-207c-4309-9b7f-483feadfdc30 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:37.454170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc5783-f5b1-447d-bb61-fd41516f9e96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:37.454786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc5783-f5b1-447d-bb61-fd41516f9e96 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:37.455072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc5783-f5b1-447d-bb61-fd41516f9e96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:37.455350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc5783-f5b1-447d-bb61-fd41516f9e96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:37.466114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc5783-f5b1-447d-bb61-fd41516f9e96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:37.467742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc5783-f5b1-447d-bb61-fd41516f9e96 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:37.471740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:37.480432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1114/2229] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:37.837639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3a85e-1a43-40c8-957d-163c8f92e9b7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:37.900972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3a85e-1a43-40c8-957d-163c8f92e9b7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:37.905813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:37.906722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:37.907009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:37.907290 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] PATH_INFO: `/v3/users/986bd53c012c49ca925577b3479706e5` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:37.908503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=986bd53c012c49ca925577b3479706e5)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:37.910539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:37.958662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '986bd53c012c49ca925577b3479706e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:37.966398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '986bd53c012c49ca925577b3479706e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:37.967092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '986bd53c012c49ca925577b3479706e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:37.979608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '986bd53c012c49ca925577b3479706e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:37.987285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '986bd53c012c49ca925577b3479706e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:37.994098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1116/2230] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:37 2022] DELETE /identity/v3/users/986bd53c012c49ca925577b3479706e5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:37.998778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbe59c-2aa7-4419-9585-abbd09c489b2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:37.999072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbe59c-2aa7-4419-9585-abbd09c489b2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:37.999351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbe59c-2aa7-4419-9585-abbd09c489b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:38.017141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-58bbe59c-2aa7-4419-9585-abbd09c489b2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:38.018329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-58bbe59c-2aa7-4419-9585-abbd09c489b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:38.077105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1115/2231] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:38.086992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:38.088427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:38.115513 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:38.149279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b09190-6dc3-4d69-90f8-961715a58933 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:38.153048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ec478-7b89-4519-bb10-3bcb8d429ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:38.153693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ec478-7b89-4519-bb10-3bcb8d429ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:38.153979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ec478-7b89-4519-bb10-3bcb8d429ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:38.154258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ec478-7b89-4519-bb10-3bcb8d429ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:38.163267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ec478-7b89-4519-bb10-3bcb8d429ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:38.164735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ec478-7b89-4519-bb10-3bcb8d429ac4 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:38.174783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1117/2232] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:38.454119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbe59c-2aa7-4419-9585-abbd09c489b2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:38.455779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-58bbe59c-2aa7-4419-9585-abbd09c489b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:38.474166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbe59c-2aa7-4419-9585-abbd09c489b2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:38.481500 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:38.483248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:38.483885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:38.484169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] PATH_INFO: `/v3/projects/fa00c52f6f714422be533a39ced08ef4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:38.485519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa00c52f6f714422be533a39ced08ef4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:38.487464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:38.537973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa00c52f6f714422be533a39ced08ef4', 'request_id': 'req-1591774c-2c98-4038-b8f4-71e61a0c36c6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:38.545962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa00c52f6f714422be533a39ced08ef4', 'request_id': 'req-1591774c-2c98-4038-b8f4-71e61a0c36c6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:38.552525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa00c52f6f714422be533a39ced08ef4', 'request_id': 'req-1591774c-2c98-4038-b8f4-71e61a0c36c6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:38.564763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] The token cache is being invalidate because project fa00c52f6f714422be533a39ced08ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:38.565050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:38.565674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:38.567198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1116/2233] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:38 2022] DELETE /identity/v3/projects/fa00c52f6f714422be533a39ced08ef4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:38.571125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ec478-7b89-4519-bb10-3bcb8d429ac4 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:38.582571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ec478-7b89-4519-bb10-3bcb8d429ac4 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:38.639539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:38.640436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:38.640718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:38.640995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] PATH_INFO: `/v3/users/5e3e0e01cb9b4a9293e43f4c86c2ba78` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:38.642197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3e0e01cb9b4a9293e43f4c86c2ba78)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:38.644120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:38.689571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3e0e01cb9b4a9293e43f4c86c2ba78'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:38.696363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3e0e01cb9b4a9293e43f4c86c2ba78'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:38.697541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3e0e01cb9b4a9293e43f4c86c2ba78'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:38.709202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3e0e01cb9b4a9293e43f4c86c2ba78'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:38.716532 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cb15399c-f4c1-486c-b74c-0a59cd2b7991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3e0e01cb9b4a9293e43f4c86c2ba78'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:38.722337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1118/2234] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:38 2022] DELETE /identity/v3/users/5e3e0e01cb9b4a9293e43f4c86c2ba78 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:38.992317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:38.994536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:39.012209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591774c-2c98-4038-b8f4-71e61a0c36c6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:39.066722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:39.067578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.067857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.068129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] PATH_INFO: `/v3/projects/754d91e615744f2b8eaa07338c3b0d15` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.069358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=754d91e615744f2b8eaa07338c3b0d15)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:39.071316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:39.095265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:39.118348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '754d91e615744f2b8eaa07338c3b0d15', 'request_id': 'req-265ba638-2222-4fbc-9f28-c13a6a7d5e35'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:39.127418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '754d91e615744f2b8eaa07338c3b0d15', 'request_id': 'req-265ba638-2222-4fbc-9f28-c13a6a7d5e35'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:39.133416 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '754d91e615744f2b8eaa07338c3b0d15', 'request_id': 'req-265ba638-2222-4fbc-9f28-c13a6a7d5e35'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:39.144433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] The token cache is being invalidate because project 754d91e615744f2b8eaa07338c3b0d15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:39.144721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:39.145368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-265ba638-2222-4fbc-9f28-c13a6a7d5e35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:39.146907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1117/2235] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:38 2022] DELETE /identity/v3/projects/754d91e615744f2b8eaa07338c3b0d15 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:39.153020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fa747-c134-437a-962b-6aec05be0e3d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.153527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fa747-c134-437a-962b-6aec05be0e3d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.153623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fa747-c134-437a-962b-6aec05be0e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.171660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d18fa747-c134-437a-962b-6aec05be0e3d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:39.172878 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d18fa747-c134-437a-962b-6aec05be0e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:39.212300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:39.232954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1119/2236] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:39.241887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5bb79-3c52-4b0d-9b29-0319dba88706 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.242118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5bb79-3c52-4b0d-9b29-0319dba88706 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.242401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5bb79-3c52-4b0d-9b29-0319dba88706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.260590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a8c5bb79-3c52-4b0d-9b29-0319dba88706 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:39.261802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a8c5bb79-3c52-4b0d-9b29-0319dba88706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:39.269719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1118/2237] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:39.273916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fa747-c134-437a-962b-6aec05be0e3d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:39.284790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fa747-c134-437a-962b-6aec05be0e3d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:39.289292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0aa742-04fd-4da3-b604-4f1416e57989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:39.290012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0aa742-04fd-4da3-b604-4f1416e57989 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.290786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0aa742-04fd-4da3-b604-4f1416e57989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.291175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0aa742-04fd-4da3-b604-4f1416e57989 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.291909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0aa742-04fd-4da3-b604-4f1416e57989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:35:39.292218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0aa742-04fd-4da3-b604-4f1416e57989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:39.293888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0aa742-04fd-4da3-b604-4f1416e57989 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:39.305639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1120/2238] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:35:39.310291 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5bb79-3c52-4b0d-9b29-0319dba88706 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:39.319447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5bb79-3c52-4b0d-9b29-0319dba88706 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:39.324729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c60981-6d78-47d7-bf88-c29e9e57a150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:39.325373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60981-6d78-47d7-bf88-c29e9e57a150 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.325655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60981-6d78-47d7-bf88-c29e9e57a150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.325927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60981-6d78-47d7-bf88-c29e9e57a150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.326642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c60981-6d78-47d7-bf88-c29e9e57a150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:39.328559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c60981-6d78-47d7-bf88-c29e9e57a150 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:39.355480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1119/2239] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:35:39 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:39.359568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0aa742-04fd-4da3-b604-4f1416e57989 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:39.369395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0aa742-04fd-4da3-b604-4f1416e57989 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:39.425919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc7c7d-7dad-4e77-864a-bbb4c4daf390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:39.426578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc7c7d-7dad-4e77-864a-bbb4c4daf390 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.426857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc7c7d-7dad-4e77-864a-bbb4c4daf390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.427129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc7c7d-7dad-4e77-864a-bbb4c4daf390 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.427809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc7c7d-7dad-4e77-864a-bbb4c4daf390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:39.429784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc7c7d-7dad-4e77-864a-bbb4c4daf390 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:39.449463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1121/2240] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:35:39 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:35:39.453664 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c60981-6d78-47d7-bf88-c29e9e57a150 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:39.464953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c60981-6d78-47d7-bf88-c29e9e57a150 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:39.469563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174e529-219c-44fd-9b3a-66792e57872f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:39.470221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174e529-219c-44fd-9b3a-66792e57872f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.470522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174e529-219c-44fd-9b3a-66792e57872f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.470779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174e529-219c-44fd-9b3a-66792e57872f admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.471504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174e529-219c-44fd-9b3a-66792e57872f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:39.473128 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174e529-219c-44fd-9b3a-66792e57872f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:39.484061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1120/2241] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:35:39.489607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc7c7d-7dad-4e77-864a-bbb4c4daf390 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:39.500023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc7c7d-7dad-4e77-864a-bbb4c4daf390 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:39.504890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a791b1-798c-4059-b9c3-a54a008c97a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:39.505553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a791b1-798c-4059-b9c3-a54a008c97a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.505843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a791b1-798c-4059-b9c3-a54a008c97a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.506130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a791b1-798c-4059-b9c3-a54a008c97a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.506857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a791b1-798c-4059-b9c3-a54a008c97a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:39.509200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a791b1-798c-4059-b9c3-a54a008c97a8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:39.519162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1122/2242] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 17:35:39.523412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174e529-219c-44fd-9b3a-66792e57872f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:39.534342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174e529-219c-44fd-9b3a-66792e57872f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:39.538815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf4a2-d432-491f-bb87-557bced1c02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:39.539702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf4a2-d432-491f-bb87-557bced1c02e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.539984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf4a2-d432-491f-bb87-557bced1c02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.540260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf4a2-d432-491f-bb87-557bced1c02e admin admin] PATH_INFO: `/v3/projects/7ec6295428384bb9923c4e7377803cc0/users/f6d7230f4f4a404e9947f5a49c455e51/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.554993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edf4a2-d432-491f-bb87-557bced1c02e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec6295428384bb9923c4e7377803cc0, user_id=f6d7230f4f4a404e9947f5a49c455e51, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:39.557762 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edf4a2-d432-491f-bb87-557bced1c02e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:39.575363 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1121/2243] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:35:39 2022] PUT /identity/v3/projects/7ec6295428384bb9923c4e7377803cc0/users/f6d7230f4f4a404e9947f5a49c455e51/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:39.585833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b2cdd-0093-46e8-8e94-02cf2811b829 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.586125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b2cdd-0093-46e8-8e94-02cf2811b829 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.586407 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b2cdd-0093-46e8-8e94-02cf2811b829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.615928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-063b2cdd-0093-46e8-8e94-02cf2811b829 None None] MFA Rules not processed for user `f6d7230f4f4a404e9947f5a49c455e51`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:39.617172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-063b2cdd-0093-46e8-8e94-02cf2811b829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:39.690997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1123/2244] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:39.703344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf4a2-d432-491f-bb87-557bced1c02e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:39.705135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e6edf4a2-d432-491f-bb87-557bced1c02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:39.762902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf4a2-d432-491f-bb87-557bced1c02e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:39.766518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fd42f-e3bb-427a-b2fd-803521bec0fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:39.767156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fd42f-e3bb-427a-b2fd-803521bec0fc service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.767436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fd42f-e3bb-427a-b2fd-803521bec0fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.767713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fd42f-e3bb-427a-b2fd-803521bec0fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.775983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fd42f-e3bb-427a-b2fd-803521bec0fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:39.777442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fd42f-e3bb-427a-b2fd-803521bec0fc service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:39.786021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1122/2245] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:39.810979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b2cdd-0093-46e8-8e94-02cf2811b829 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:39.854066 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:39.874244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b2cdd-0093-46e8-8e94-02cf2811b829 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:39.878840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:39.879708 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.879988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.880262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] PATH_INFO: `/v3/users/f6d7230f4f4a404e9947f5a49c455e51` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.881492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d7230f4f4a404e9947f5a49c455e51)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:39.883421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:39.928540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d7230f4f4a404e9947f5a49c455e51'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:39.935667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d7230f4f4a404e9947f5a49c455e51'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:39.936798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d7230f4f4a404e9947f5a49c455e51'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:39.948596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d7230f4f4a404e9947f5a49c455e51'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:39.955841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d7230f4f4a404e9947f5a49c455e51'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:39.962216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1124/2246] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:35:39 2022] DELETE /identity/v3/users/f6d7230f4f4a404e9947f5a49c455e51 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:35:39.966767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb01dca-4618-4afc-8e68-e760ade5bbb2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:39.967056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb01dca-4618-4afc-8e68-e760ade5bbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:39.967337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb01dca-4618-4afc-8e68-e760ade5bbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:39.986738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-9bb01dca-4618-4afc-8e68-e760ade5bbb2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:35:39.988205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9bb01dca-4618-4afc-8e68-e760ade5bbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:40.034854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:40.051745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1123/2247] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:35:40.061631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:40.063083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:40.123597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce83ddb-1317-4198-9c9f-e23ea740492e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:40.127220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb891c-621d-4903-ad44-63a8ee1a1b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:40.127863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb891c-621d-4903-ad44-63a8ee1a1b41 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:40.128146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb891c-621d-4903-ad44-63a8ee1a1b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:40.128462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb891c-621d-4903-ad44-63a8ee1a1b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:40.138159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb891c-621d-4903-ad44-63a8ee1a1b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:40.139592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb891c-621d-4903-ad44-63a8ee1a1b41 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:40.149039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1125/2248] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:35:40.622292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb01dca-4618-4afc-8e68-e760ade5bbb2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:35:40.623770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9bb01dca-4618-4afc-8e68-e760ade5bbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:35:40.638960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb01dca-4618-4afc-8e68-e760ade5bbb2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:35:40.644835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:35:40.645598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:35:40.645877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:35:40.646132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] PATH_INFO: `/v3/projects/7ec6295428384bb9923c4e7377803cc0` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:35:40.647328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec6295428384bb9923c4e7377803cc0)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:35:40.649625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:35:40.695827 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec6295428384bb9923c4e7377803cc0', 'request_id': 'req-5aed4097-db51-48a0-86eb-9639fb9dcd13'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:40.704787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec6295428384bb9923c4e7377803cc0', 'request_id': 'req-5aed4097-db51-48a0-86eb-9639fb9dcd13'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:40.710499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec6295428384bb9923c4e7377803cc0', 'request_id': 'req-5aed4097-db51-48a0-86eb-9639fb9dcd13'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:40.721653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] The token cache is being invalidate because project 7ec6295428384bb9923c4e7377803cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:35:40.721941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:40.722585 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:35:40.724097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1124/2249] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:35:40 2022] DELETE /identity/v3/projects/7ec6295428384bb9923c4e7377803cc0 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 17:35:50.144816 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:35:50.718605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:38:02.787124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb891c-621d-4903-ad44-63a8ee1a1b41 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:38:02.788570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1bdb891c-621d-4903-ad44-63a8ee1a1b41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:38:02.856781 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb891c-621d-4903-ad44-63a8ee1a1b41 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:38:02.913357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410ff7e-477c-41df-a0a9-73747ddffbc2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:38:02.914005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410ff7e-477c-41df-a0a9-73747ddffbc2 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:38:02.914285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410ff7e-477c-41df-a0a9-73747ddffbc2 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:38:02.914560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410ff7e-477c-41df-a0a9-73747ddffbc2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:38:02.916355 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6410ff7e-477c-41df-a0a9-73747ddffbc2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:38:02.928260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410ff7e-477c-41df-a0a9-73747ddffbc2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:38:02.929899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410ff7e-477c-41df-a0a9-73747ddffbc2 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:38:02.941064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1126/2250] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:38:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:38:12.936540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=58 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:39:06.087732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:39:06.089274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:39:06.147261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed4097-db51-48a0-86eb-9639fb9dcd13 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:39:06.151574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea092da4-5250-4790-849a-e3be69ad618a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:39:06.152222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea092da4-5250-4790-849a-e3be69ad618a service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:39:06.152540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea092da4-5250-4790-849a-e3be69ad618a service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:39:06.152821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea092da4-5250-4790-849a-e3be69ad618a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:39:06.154624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ea092da4-5250-4790-849a-e3be69ad618a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:39:06.213911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea092da4-5250-4790-849a-e3be69ad618a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:39:06.215341 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea092da4-5250-4790-849a-e3be69ad618a service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:39:06.224279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1125/2251] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:39:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:39:14.762886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410ff7e-477c-41df-a0a9-73747ddffbc2 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:39:14.772071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410ff7e-477c-41df-a0a9-73747ddffbc2 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:39:14.776241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37399348-4acf-4dc4-97be-7ce9e11fd245 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:39:14.776913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37399348-4acf-4dc4-97be-7ce9e11fd245 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:39:14.777201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37399348-4acf-4dc4-97be-7ce9e11fd245 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:39:14.777499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37399348-4acf-4dc4-97be-7ce9e11fd245 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:39:14.779255 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-37399348-4acf-4dc4-97be-7ce9e11fd245 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:39:14.839931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37399348-4acf-4dc4-97be-7ce9e11fd245 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:39:14.841422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37399348-4acf-4dc4-97be-7ce9e11fd245 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:39:14.850984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1127/2252] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:39:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:39:16.220223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=48 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:39:19.112079 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea092da4-5250-4790-849a-e3be69ad618a service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:39:19.120582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea092da4-5250-4790-849a-e3be69ad618a service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:39:19.124441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3469126-bf4d-4e17-91a0-1595718a6935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:39:19.125111 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3469126-bf4d-4e17-91a0-1595718a6935 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:39:19.125434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3469126-bf4d-4e17-91a0-1595718a6935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:39:19.125745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3469126-bf4d-4e17-91a0-1595718a6935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:39:19.127528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a3469126-bf4d-4e17-91a0-1595718a6935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:39:19.187256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3469126-bf4d-4e17-91a0-1595718a6935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:39:19.188736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3469126-bf4d-4e17-91a0-1595718a6935 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:39:19.198478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1126/2253] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:39:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:39:24.846845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=28 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:39:29.193879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=28 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:39:54.472568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37399348-4acf-4dc4-97be-7ce9e11fd245 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:39:54.481240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37399348-4acf-4dc4-97be-7ce9e11fd245 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:39:54.484792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ff886-2d81-4b53-a62c-456573f6a775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:39:54.485424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ff886-2d81-4b53-a62c-456573f6a775 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:39:54.485705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ff886-2d81-4b53-a62c-456573f6a775 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:39:54.485980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ff886-2d81-4b53-a62c-456573f6a775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:39:54.487718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d98ff886-2d81-4b53-a62c-456573f6a775 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:39:54.546478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ff886-2d81-4b53-a62c-456573f6a775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:39:54.547926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ff886-2d81-4b53-a62c-456573f6a775 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:39:54.556822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1128/2254] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:39:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:40:01.477185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3469126-bf4d-4e17-91a0-1595718a6935 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:40:01.478651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a3469126-bf4d-4e17-91a0-1595718a6935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:40:01.534727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3469126-bf4d-4e17-91a0-1595718a6935 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:40:01.538331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ba331-c318-4471-ba9d-c1a2c4fd066f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:40:01.538974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba331-c318-4471-ba9d-c1a2c4fd066f service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:40:01.539260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba331-c318-4471-ba9d-c1a2c4fd066f service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:40:01.539537 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba331-c318-4471-ba9d-c1a2c4fd066f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:40:01.541323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-282ba331-c318-4471-ba9d-c1a2c4fd066f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:40:01.551654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ba331-c318-4471-ba9d-c1a2c4fd066f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:40:01.553103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ba331-c318-4471-ba9d-c1a2c4fd066f service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:40:01.561101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1127/2255] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:40:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:40:04.552753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=28 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:40:11.557436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=26 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:40:12.818419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ff886-2d81-4b53-a62c-456573f6a775 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:40:12.827533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ff886-2d81-4b53-a62c-456573f6a775 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:40:12.831156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81904189-d27e-44fe-bf42-45fa4e8d2c6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:40:12.831794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904189-d27e-44fe-bf42-45fa4e8d2c6d service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:40:12.832080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904189-d27e-44fe-bf42-45fa4e8d2c6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:40:12.832392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904189-d27e-44fe-bf42-45fa4e8d2c6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:40:12.834136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-81904189-d27e-44fe-bf42-45fa4e8d2c6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:40:12.892799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81904189-d27e-44fe-bf42-45fa4e8d2c6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:40:12.894220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81904189-d27e-44fe-bf42-45fa4e8d2c6d service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:40:12.903696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1129/2256] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:40:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:40:22.898868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=28 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:41:59.388530 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ba331-c318-4471-ba9d-c1a2c4fd066f service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:41:59.407194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ba331-c318-4471-ba9d-c1a2c4fd066f service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:41:59.410833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41de66-a52a-4720-9fba-b9fc349d5fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:41:59.411470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41de66-a52a-4720-9fba-b9fc349d5fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:41:59.411751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41de66-a52a-4720-9fba-b9fc349d5fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:41:59.412028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41de66-a52a-4720-9fba-b9fc349d5fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:41:59.413796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8e41de66-a52a-4720-9fba-b9fc349d5fa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:41:59.538377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41de66-a52a-4720-9fba-b9fc349d5fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:41:59.539721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41de66-a52a-4720-9fba-b9fc349d5fa0 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:41:59.560492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1128/2257] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:41:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:42:00.740909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81904189-d27e-44fe-bf42-45fa4e8d2c6d service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:00.770564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81904189-d27e-44fe-bf42-45fa4e8d2c6d service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:00.774628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ea307-dab0-4604-9712-124ecf29c2c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:00.775343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ea307-dab0-4604-9712-124ecf29c2c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:00.775696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ea307-dab0-4604-9712-124ecf29c2c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:00.776040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ea307-dab0-4604-9712-124ecf29c2c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:00.778549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dd9ea307-dab0-4604-9712-124ecf29c2c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:00.792920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ea307-dab0-4604-9712-124ecf29c2c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:00.794655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ea307-dab0-4604-9712-124ecf29c2c3 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:00.811549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1130/2258] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:42:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:42:00.825736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41de66-a52a-4720-9fba-b9fc349d5fa0 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:00.829156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8e41de66-a52a-4720-9fba-b9fc349d5fa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:00.852379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41de66-a52a-4720-9fba-b9fc349d5fa0 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:00.862465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579afccd-9ed4-49a1-b501-5aafe37637e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:00.863140 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579afccd-9ed4-49a1-b501-5aafe37637e3 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:00.863427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579afccd-9ed4-49a1-b501-5aafe37637e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:00.863705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579afccd-9ed4-49a1-b501-5aafe37637e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:00.865730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-579afccd-9ed4-49a1-b501-5aafe37637e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:00.881989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579afccd-9ed4-49a1-b501-5aafe37637e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:00.883109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579afccd-9ed4-49a1-b501-5aafe37637e3 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:00.904501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1129/2259] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:42:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:42:04.612160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17d342-7d16-4593-a3a3-a529faeeaa2d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:04.612795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17d342-7d16-4593-a3a3-a529faeeaa2d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:04.612795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17d342-7d16-4593-a3a3-a529faeeaa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:04.631251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-dc17d342-7d16-4593-a3a3-a529faeeaa2d None None] MFA Rules not processed for user `fa82bddf9aa0455797d42993fe850ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:42:04.632489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dc17d342-7d16-4593-a3a3-a529faeeaa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:04.640083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1131/2260] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:42:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:42:04.946439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579afccd-9ed4-49a1-b501-5aafe37637e3 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:04.956135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579afccd-9ed4-49a1-b501-5aafe37637e3 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:04.959700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c4a7e-f853-4a86-bc22-241f7065a96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:04.960360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c4a7e-f853-4a86-bc22-241f7065a96e service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:04.960644 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c4a7e-f853-4a86-bc22-241f7065a96e service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:04.960921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c4a7e-f853-4a86-bc22-241f7065a96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:04.962678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-283c4a7e-f853-4a86-bc22-241f7065a96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:04.973083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c4a7e-f853-4a86-bc22-241f7065a96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:04.974511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c4a7e-f853-4a86-bc22-241f7065a96e service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:04.982619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1130/2261] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:42:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:42:05.296271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17d342-7d16-4593-a3a3-a529faeeaa2d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:05.297740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dc17d342-7d16-4593-a3a3-a529faeeaa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:05.335777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17d342-7d16-4593-a3a3-a529faeeaa2d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:05.340876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735351f-78db-495d-959d-eecededb4575 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:05.341524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735351f-78db-495d-959d-eecededb4575 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:05.341803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735351f-78db-495d-959d-eecededb4575 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:05.342080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735351f-78db-495d-959d-eecededb4575 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:05.344056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9735351f-78db-495d-959d-eecededb4575 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:05.355561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735351f-78db-495d-959d-eecededb4575 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:05.357257 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735351f-78db-495d-959d-eecededb4575 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:05.366947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1132/2262] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:42:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:42:08.716902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c4a7e-f853-4a86-bc22-241f7065a96e service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:08.718798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-283c4a7e-f853-4a86-bc22-241f7065a96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:08.774205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:42:08.835276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c4a7e-f853-4a86-bc22-241f7065a96e service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:08.840028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:08.841075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:08.841808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:08.842280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] PATH_INFO: `/v3/users/626ba8ee94d24de1839d4f3471bbfe8f` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:08.843523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=626ba8ee94d24de1839d4f3471bbfe8f)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:08.845523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:08.899598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '626ba8ee94d24de1839d4f3471bbfe8f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:42:08.900419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '626ba8ee94d24de1839d4f3471bbfe8f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:42:08.901306 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '626ba8ee94d24de1839d4f3471bbfe8f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:42:08.909111 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '626ba8ee94d24de1839d4f3471bbfe8f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:42:08.923726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '626ba8ee94d24de1839d4f3471bbfe8f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:42:08.946320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1131/2263] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:42:08 2022] DELETE /identity/v3/users/626ba8ee94d24de1839d4f3471bbfe8f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:42:08.951072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61be1c-7190-411b-88ca-76e09cb7e306 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:08.951575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61be1c-7190-411b-88ca-76e09cb7e306 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:08.951686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61be1c-7190-411b-88ca-76e09cb7e306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:08.978537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2a61be1c-7190-411b-88ca-76e09cb7e306 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:42:08.979741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2a61be1c-7190-411b-88ca-76e09cb7e306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:09.038760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1133/2264] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:42:09.048473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:09.049873 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:09.106873 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5770b1-3f02-43d0-be7a-d161de829f8b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:09.110484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260a3b8-5cd4-40be-b315-c03a06608182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:09.111131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260a3b8-5cd4-40be-b315-c03a06608182 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:09.111417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260a3b8-5cd4-40be-b315-c03a06608182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:09.111697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260a3b8-5cd4-40be-b315-c03a06608182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:09.120754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260a3b8-5cd4-40be-b315-c03a06608182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:09.122818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260a3b8-5cd4-40be-b315-c03a06608182 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:09.132390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1132/2265] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:42:09.472324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61be1c-7190-411b-88ca-76e09cb7e306 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:09.473746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2a61be1c-7190-411b-88ca-76e09cb7e306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:09.486752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61be1c-7190-411b-88ca-76e09cb7e306 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:09.491684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:09.492732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:09.493019 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:09.493298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] PATH_INFO: `/v3/projects/88dd2b2db7b846ff99d3091575fa0b04` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:09.494639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=88dd2b2db7b846ff99d3091575fa0b04)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:09.496595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:09.546668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88dd2b2db7b846ff99d3091575fa0b04', 'request_id': 'req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:42:09.553078 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88dd2b2db7b846ff99d3091575fa0b04', 'request_id': 'req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:42:09.561710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88dd2b2db7b846ff99d3091575fa0b04', 'request_id': 'req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:42:09.573070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] The token cache is being invalidate because project 88dd2b2db7b846ff99d3091575fa0b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:42:09.573358 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:42:09.573989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ef4bfc0-a2f3-4307-b451-b1b53cd4af5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:42:09.576067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1134/2266] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:42:09 2022] DELETE /identity/v3/projects/88dd2b2db7b846ff99d3091575fa0b04 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:42:09.582243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4812b0-713a-48d9-ac79-42609cc24b79 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:09.582609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4812b0-713a-48d9-ac79-42609cc24b79 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:09.583013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4812b0-713a-48d9-ac79-42609cc24b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:09.630319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1d4812b0-713a-48d9-ac79-42609cc24b79 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:42:09.631518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1d4812b0-713a-48d9-ac79-42609cc24b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:09.640810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:42:09.703241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1133/2267] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:42:09.711955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbc352-3d48-4c45-9f25-a629129acbd8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:09.712434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbc352-3d48-4c45-9f25-a629129acbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:09.712672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbc352-3d48-4c45-9f25-a629129acbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:09.732892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-79bbc352-3d48-4c45-9f25-a629129acbd8 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:42:09.734101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-79bbc352-3d48-4c45-9f25-a629129acbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:09.742561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1135/2268] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:42:09.746492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4812b0-713a-48d9-ac79-42609cc24b79 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:09.756992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4812b0-713a-48d9-ac79-42609cc24b79 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:09.761844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397d776-ad31-4535-9608-5841a6610fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:09.762552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397d776-ad31-4535-9608-5841a6610fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:09.763368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397d776-ad31-4535-9608-5841a6610fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:09.763649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397d776-ad31-4535-9608-5841a6610fdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:09.764554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397d776-ad31-4535-9608-5841a6610fdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:42:09.764872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397d776-ad31-4535-9608-5841a6610fdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:09.766480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397d776-ad31-4535-9608-5841a6610fdc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:09.778345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1134/2269] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:42:09.782731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbc352-3d48-4c45-9f25-a629129acbd8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:09.791913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbc352-3d48-4c45-9f25-a629129acbd8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:09.796815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e114f-dec9-40c5-9c2f-c5e9c795e233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:09.797472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e114f-dec9-40c5-9c2f-c5e9c795e233 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:09.797758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e114f-dec9-40c5-9c2f-c5e9c795e233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:09.798038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e114f-dec9-40c5-9c2f-c5e9c795e233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:09.798724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e114f-dec9-40c5-9c2f-c5e9c795e233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:09.800669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e114f-dec9-40c5-9c2f-c5e9c795e233 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:09.803920 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:42:09.828914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1136/2270] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:42:09 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:42:09.833135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397d776-ad31-4535-9608-5841a6610fdc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:09.843071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397d776-ad31-4535-9608-5841a6610fdc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:09.898785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa7fdd-6f27-4a48-b258-a37f7bf4ed8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:09.899455 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa7fdd-6f27-4a48-b258-a37f7bf4ed8c admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:09.899735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa7fdd-6f27-4a48-b258-a37f7bf4ed8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:09.900010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa7fdd-6f27-4a48-b258-a37f7bf4ed8c admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:09.900730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa7fdd-6f27-4a48-b258-a37f7bf4ed8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:09.902618 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa7fdd-6f27-4a48-b258-a37f7bf4ed8c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:09.929924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1135/2271] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:42:09 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:42:09.934090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e114f-dec9-40c5-9c2f-c5e9c795e233 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:09.947496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e114f-dec9-40c5-9c2f-c5e9c795e233 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:09.952843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ec820-7bc5-4f20-8c82-3254aea18e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:09.953522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ec820-7bc5-4f20-8c82-3254aea18e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:09.953840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ec820-7bc5-4f20-8c82-3254aea18e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:09.954120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ec820-7bc5-4f20-8c82-3254aea18e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:09.954871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ec820-7bc5-4f20-8c82-3254aea18e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:09.956518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ec820-7bc5-4f20-8c82-3254aea18e7d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:09.967520 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1137/2272] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:42:09.971421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa7fdd-6f27-4a48-b258-a37f7bf4ed8c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:09.983487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa7fdd-6f27-4a48-b258-a37f7bf4ed8c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:09.988673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3417b-6a99-484f-89b5-d2f2eff64891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:09.989401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3417b-6a99-484f-89b5-d2f2eff64891 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:09.989694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3417b-6a99-484f-89b5-d2f2eff64891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:09.989975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3417b-6a99-484f-89b5-d2f2eff64891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:09.990705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3417b-6a99-484f-89b5-d2f2eff64891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:09.992317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3417b-6a99-484f-89b5-d2f2eff64891 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:10.004958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1136/2273] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:42:10.008810 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ec820-7bc5-4f20-8c82-3254aea18e7d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:10.026102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ec820-7bc5-4f20-8c82-3254aea18e7d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:10.030940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc9430-258b-4cf6-aa80-3d0530e1e1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:10.032068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc9430-258b-4cf6-aa80-3d0530e1e1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:10.032386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc9430-258b-4cf6-aa80-3d0530e1e1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:10.032672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc9430-258b-4cf6-aa80-3d0530e1e1b1 admin admin] PATH_INFO: `/v3/projects/66ee4df184ae4eb6a9a559fd40a7ecc1/users/e34d1c2cd15b47b39e8273587adc205f/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:10.050892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc9430-258b-4cf6-aa80-3d0530e1e1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ee4df184ae4eb6a9a559fd40a7ecc1, user_id=e34d1c2cd15b47b39e8273587adc205f, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:10.053624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc9430-258b-4cf6-aa80-3d0530e1e1b1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:10.069618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1138/2274] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:42:10 2022] PUT /identity/v3/projects/66ee4df184ae4eb6a9a559fd40a7ecc1/users/e34d1c2cd15b47b39e8273587adc205f/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:42:10.083042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c8fd6-6044-442b-89c6-249965c79d73 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:10.083343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c8fd6-6044-442b-89c6-249965c79d73 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:10.083630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c8fd6-6044-442b-89c6-249965c79d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:10.110701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b42c8fd6-6044-442b-89c6-249965c79d73 None None] MFA Rules not processed for user `e34d1c2cd15b47b39e8273587adc205f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:42:10.111956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b42c8fd6-6044-442b-89c6-249965c79d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:10.174505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1137/2275] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:42:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:42:10.186538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc9430-258b-4cf6-aa80-3d0530e1e1b1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:10.188468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a6fc9430-258b-4cf6-aa80-3d0530e1e1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:10.243870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc9430-258b-4cf6-aa80-3d0530e1e1b1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:10.248165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13a92d-5041-4606-995f-561247cfde61 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e07c59babad479cadbf74f15ee5eea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:10.248844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a92d-5041-4606-995f-561247cfde61 service glance] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:10.249127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a92d-5041-4606-995f-561247cfde61 service glance] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:10.249404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a92d-5041-4606-995f-561247cfde61 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:10.257681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13a92d-5041-4606-995f-561247cfde61 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:10.259113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13a92d-5041-4606-995f-561247cfde61 service glance] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:10.267799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1139/2276] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:42:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:42:14.061634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c8fd6-6044-442b-89c6-249965c79d73 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:14.063441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b42c8fd6-6044-442b-89c6-249965c79d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:14.075531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:42:14.143829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c8fd6-6044-442b-89c6-249965c79d73 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:14.149388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d462c2-af9f-4c9f-aa3d-4d234e5ac6ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:14.150105 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d462c2-af9f-4c9f-aa3d-4d234e5ac6ae service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:14.150387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d462c2-af9f-4c9f-aa3d-4d234e5ac6ae service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:14.150663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d462c2-af9f-4c9f-aa3d-4d234e5ac6ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:14.152836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-51d462c2-af9f-4c9f-aa3d-4d234e5ac6ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:14.165553 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d462c2-af9f-4c9f-aa3d-4d234e5ac6ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:14.167130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d462c2-af9f-4c9f-aa3d-4d234e5ac6ae service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:14.178436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1138/2277] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:42:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:42:18.212672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13a92d-5041-4606-995f-561247cfde61 service glance] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:42:18.214286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ce13a92d-5041-4606-995f-561247cfde61 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:18.236511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13a92d-5041-4606-995f-561247cfde61 service glance] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:42:18.242004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee7aaf-6485-407d-8996-aecf3599e2ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:42:18.242722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee7aaf-6485-407d-8996-aecf3599e2ce service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:42:18.243024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee7aaf-6485-407d-8996-aecf3599e2ce service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:42:18.243603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee7aaf-6485-407d-8996-aecf3599e2ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:42:18.245435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-61ee7aaf-6485-407d-8996-aecf3599e2ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:42:18.258660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee7aaf-6485-407d-8996-aecf3599e2ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:42:18.260169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee7aaf-6485-407d-8996-aecf3599e2ce service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:42:18.270690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1140/2278] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:42:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:42:24.173751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=30 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:42:28.266322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:43:06.752717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d462c2-af9f-4c9f-aa3d-4d234e5ac6ae service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:43:06.754099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-51d462c2-af9f-4c9f-aa3d-4d234e5ac6ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:43:06.821370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d462c2-af9f-4c9f-aa3d-4d234e5ac6ae service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:43:06.826624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f9895-82d3-4ba8-850c-f046058ad7f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:43:06.827274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f9895-82d3-4ba8-850c-f046058ad7f9 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:43:06.827557 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f9895-82d3-4ba8-850c-f046058ad7f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:43:06.827835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f9895-82d3-4ba8-850c-f046058ad7f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:43:06.829725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2d9f9895-82d3-4ba8-850c-f046058ad7f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:43:06.841424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f9895-82d3-4ba8-850c-f046058ad7f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:43:06.843027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f9895-82d3-4ba8-850c-f046058ad7f9 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:43:06.852797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1139/2279] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:43:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:43:16.848633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=32 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:44:06.021879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee7aaf-6485-407d-8996-aecf3599e2ce service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:44:06.021879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-61ee7aaf-6485-407d-8996-aecf3599e2ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:44:06.103365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee7aaf-6485-407d-8996-aecf3599e2ce service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:44:06.107208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a8afa-5eef-4a15-a70f-b63125183062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:44:06.107860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a8afa-5eef-4a15-a70f-b63125183062 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:44:06.108150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a8afa-5eef-4a15-a70f-b63125183062 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:44:06.108474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a8afa-5eef-4a15-a70f-b63125183062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:44:06.110265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-804a8afa-5eef-4a15-a70f-b63125183062 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:44:06.192091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a8afa-5eef-4a15-a70f-b63125183062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:44:06.193592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a8afa-5eef-4a15-a70f-b63125183062 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:44:06.213565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1141/2280] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:44:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:44:15.056360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f9895-82d3-4ba8-850c-f046058ad7f9 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:44:15.068935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f9895-82d3-4ba8-850c-f046058ad7f9 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:44:15.074824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2850c-bf69-4176-8288-b9ed30da3991 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:44:15.075468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2850c-bf69-4176-8288-b9ed30da3991 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:44:15.075750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2850c-bf69-4176-8288-b9ed30da3991 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:44:15.076027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2850c-bf69-4176-8288-b9ed30da3991 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:44:15.077851 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d8c2850c-bf69-4176-8288-b9ed30da3991 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:44:15.157915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2850c-bf69-4176-8288-b9ed30da3991 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:44:15.159413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2850c-bf69-4176-8288-b9ed30da3991 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:44:15.177529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1140/2281] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:44:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:44:16.210761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=70 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:44:19.069794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a8afa-5eef-4a15-a70f-b63125183062 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:44:19.078746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a8afa-5eef-4a15-a70f-b63125183062 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:44:19.082934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bfe99-489b-45fc-bc5f-66b802ea1378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:44:19.083577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bfe99-489b-45fc-bc5f-66b802ea1378 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:44:19.083863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bfe99-489b-45fc-bc5f-66b802ea1378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:44:19.084143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bfe99-489b-45fc-bc5f-66b802ea1378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:44:19.085958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3c2bfe99-489b-45fc-bc5f-66b802ea1378 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:44:19.171417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bfe99-489b-45fc-bc5f-66b802ea1378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:44:19.172915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bfe99-489b-45fc-bc5f-66b802ea1378 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:44:19.191670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1142/2282] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:44:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:44:25.174857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=41 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:44:29.188526 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=41 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:44:54.459291 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2850c-bf69-4176-8288-b9ed30da3991 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:44:54.468519 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2850c-bf69-4176-8288-b9ed30da3991 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:44:54.472523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed000-56d6-4f19-8c86-fe9cad5d90ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:44:54.473169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed000-56d6-4f19-8c86-fe9cad5d90ca service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:44:54.473448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed000-56d6-4f19-8c86-fe9cad5d90ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:44:54.473718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed000-56d6-4f19-8c86-fe9cad5d90ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:44:54.475499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fc8ed000-56d6-4f19-8c86-fe9cad5d90ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:44:54.554541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ed000-56d6-4f19-8c86-fe9cad5d90ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:44:54.556025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ed000-56d6-4f19-8c86-fe9cad5d90ca service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:44:54.574426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1141/2283] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:44:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:45:04.571850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=41 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:45:06.469242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bfe99-489b-45fc-bc5f-66b802ea1378 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:45:06.471374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3c2bfe99-489b-45fc-bc5f-66b802ea1378 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:45:06.548347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bfe99-489b-45fc-bc5f-66b802ea1378 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:45:06.551958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7aa4ee-8655-492d-b4d1-148ceebec0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:45:06.552641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7aa4ee-8655-492d-b4d1-148ceebec0ce service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:45:06.552927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7aa4ee-8655-492d-b4d1-148ceebec0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:45:06.553208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7aa4ee-8655-492d-b4d1-148ceebec0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:45:06.554984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fd7aa4ee-8655-492d-b4d1-148ceebec0ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:45:06.564572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7aa4ee-8655-492d-b4d1-148ceebec0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:45:06.566005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7aa4ee-8655-492d-b4d1-148ceebec0ce service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:45:06.573852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1143/2284] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:45:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:45:16.570408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=35 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:47:09.632842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed000-56d6-4f19-8c86-fe9cad5d90ca service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:47:09.641984 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed000-56d6-4f19-8c86-fe9cad5d90ca service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:47:09.645724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff63a-5dbd-485a-9e83-582f660b7c2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:47:09.646364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff63a-5dbd-485a-9e83-582f660b7c2a service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:47:09.646715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff63a-5dbd-485a-9e83-582f660b7c2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:47:09.646904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff63a-5dbd-485a-9e83-582f660b7c2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:47:09.655552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff63a-5dbd-485a-9e83-582f660b7c2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:47:09.657163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff63a-5dbd-485a-9e83-582f660b7c2a service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:47:09.666778 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1142/2285] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:47:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:47:19.662546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:48:08.831178 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7aa4ee-8655-492d-b4d1-148ceebec0ce service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:48:08.839538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7aa4ee-8655-492d-b4d1-148ceebec0ce service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:48:08.843812 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a20c08-63d0-4532-8167-0777f0386fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:48:08.844480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a20c08-63d0-4532-8167-0777f0386fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:48:08.844765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a20c08-63d0-4532-8167-0777f0386fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:48:08.845040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a20c08-63d0-4532-8167-0777f0386fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:48:08.846885 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-75a20c08-63d0-4532-8167-0777f0386fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:48:08.860926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a20c08-63d0-4532-8167-0777f0386fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:48:08.862351 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a20c08-63d0-4532-8167-0777f0386fb6 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:48:08.873516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1144/2286] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:48:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:48:13.771841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff63a-5dbd-485a-9e83-582f660b7c2a service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:48:13.773271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a9eff63a-5dbd-485a-9e83-582f660b7c2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:48:13.787559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff63a-5dbd-485a-9e83-582f660b7c2a service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:48:13.792854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c7ff9-f3f1-4d01-80f8-35ba8d7840e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:48:13.793605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7ff9-f3f1-4d01-80f8-35ba8d7840e5 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:48:13.793893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7ff9-f3f1-4d01-80f8-35ba8d7840e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:48:13.794178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7ff9-f3f1-4d01-80f8-35ba8d7840e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:48:13.803325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c7ff9-f3f1-4d01-80f8-35ba8d7840e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:48:13.804936 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c7ff9-f3f1-4d01-80f8-35ba8d7840e5 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:48:13.815921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1143/2287] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:48:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:48:18.868289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:48:23.811101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:48:48.571202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a20c08-63d0-4532-8167-0777f0386fb6 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:48:48.580251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a20c08-63d0-4532-8167-0777f0386fb6 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:48:48.583952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a5a7f-b279-46d6-8316-a4dab0bc8b3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:48:48.584640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a5a7f-b279-46d6-8316-a4dab0bc8b3b service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:48:48.584924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a5a7f-b279-46d6-8316-a4dab0bc8b3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:48:48.585198 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a5a7f-b279-46d6-8316-a4dab0bc8b3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:48:48.586993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3f0a5a7f-b279-46d6-8316-a4dab0bc8b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:48:48.601253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a5a7f-b279-46d6-8316-a4dab0bc8b3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:48:48.603046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a5a7f-b279-46d6-8316-a4dab0bc8b3b service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:48:48.617651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1145/2288] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:48:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:48:55.422335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c7ff9-f3f1-4d01-80f8-35ba8d7840e5 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:48:55.437038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c7ff9-f3f1-4d01-80f8-35ba8d7840e5 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:48:55.443183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0dfe5-1d29-4fc3-adab-47252066c934 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:48:55.443955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0dfe5-1d29-4fc3-adab-47252066c934 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:48:55.444520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0dfe5-1d29-4fc3-adab-47252066c934 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:48:55.444915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0dfe5-1d29-4fc3-adab-47252066c934 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:48:55.447277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-adb0dfe5-1d29-4fc3-adab-47252066c934 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:48:55.473411 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0dfe5-1d29-4fc3-adab-47252066c934 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:48:55.475382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0dfe5-1d29-4fc3-adab-47252066c934 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:48:55.488269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1144/2289] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:48:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:48:58.612594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:49:00.233494 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2449b4c-b3b4-4735-aed2-f86c217e6041 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:49:00.234071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2449b4c-b3b4-4735-aed2-f86c217e6041 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:49:00.234071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2449b4c-b3b4-4735-aed2-f86c217e6041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:49:00.262032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a2449b4c-b3b4-4735-aed2-f86c217e6041 None None] MFA Rules not processed for user `fa82bddf9aa0455797d42993fe850ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:49:00.263238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a2449b4c-b3b4-4735-aed2-f86c217e6041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:49:00.271263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1146/2290] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:49:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:49:00.579872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0dfe5-1d29-4fc3-adab-47252066c934 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:49:00.590083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0dfe5-1d29-4fc3-adab-47252066c934 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:49:00.593684 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dfbd8-bc3d-4270-bd73-9bb512c9b9d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:49:00.594324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dfbd8-bc3d-4270-bd73-9bb512c9b9d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:49:00.594608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dfbd8-bc3d-4270-bd73-9bb512c9b9d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:49:00.594883 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dfbd8-bc3d-4270-bd73-9bb512c9b9d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:49:00.597752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1a5dfbd8-bc3d-4270-bd73-9bb512c9b9d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:49:00.607413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5dfbd8-bc3d-4270-bd73-9bb512c9b9d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:49:00.608877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5dfbd8-bc3d-4270-bd73-9bb512c9b9d8 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:49:00.617523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1145/2291] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:49:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:49:03.028954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2449b4c-b3b4-4735-aed2-f86c217e6041 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:49:03.041152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2449b4c-b3b4-4735-aed2-f86c217e6041 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:49:03.046252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bab69b-db1b-4850-b340-d113b9bf4002 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:49:03.046890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bab69b-db1b-4850-b340-d113b9bf4002 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:49:03.047365 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bab69b-db1b-4850-b340-d113b9bf4002 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:49:03.047641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bab69b-db1b-4850-b340-d113b9bf4002 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:49:03.049452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-54bab69b-db1b-4850-b340-d113b9bf4002 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:49:03.061104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bab69b-db1b-4850-b340-d113b9bf4002 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:49:03.062862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bab69b-db1b-4850-b340-d113b9bf4002 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:49:03.074544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1147/2292] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:49:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:49:06.342097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dfbd8-bc3d-4270-bd73-9bb512c9b9d8 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:49:06.357460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dfbd8-bc3d-4270-bd73-9bb512c9b9d8 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:49:06.364602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce04445-f0bc-4831-af8b-57863985d751 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:49:06.365256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04445-f0bc-4831-af8b-57863985d751 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:49:06.365542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04445-f0bc-4831-af8b-57863985d751 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:49:06.365819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04445-f0bc-4831-af8b-57863985d751 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:49:06.374303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce04445-f0bc-4831-af8b-57863985d751 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:49:06.375875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce04445-f0bc-4831-af8b-57863985d751 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:49:06.385598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1146/2293] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:49:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:49:13.069615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=12 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:49:14.177117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bab69b-db1b-4850-b340-d113b9bf4002 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:49:14.189277 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bab69b-db1b-4850-b340-d113b9bf4002 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:49:14.195283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45dcdc-6c64-4fb9-b591-530cff344a44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:49:14.195916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45dcdc-6c64-4fb9-b591-530cff344a44 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:49:14.196199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45dcdc-6c64-4fb9-b591-530cff344a44 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:49:14.196516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45dcdc-6c64-4fb9-b591-530cff344a44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:49:14.204384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45dcdc-6c64-4fb9-b591-530cff344a44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:49:14.206536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45dcdc-6c64-4fb9-b591-530cff344a44 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:49:14.216136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1148/2294] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:49:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:49:14.429919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce04445-f0bc-4831-af8b-57863985d751 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:49:14.443464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce04445-f0bc-4831-af8b-57863985d751 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:49:14.449379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff6e7-9395-427a-9903-93b70eabe92a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:49:14.450072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff6e7-9395-427a-9903-93b70eabe92a service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:49:14.450766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff6e7-9395-427a-9903-93b70eabe92a service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:49:14.451082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ff6e7-9395-427a-9903-93b70eabe92a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:49:14.468569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ff6e7-9395-427a-9903-93b70eabe92a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:49:14.470236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ff6e7-9395-427a-9903-93b70eabe92a service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:49:14.482099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1147/2295] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:49:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:49:18.091735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45dcdc-6c64-4fb9-b591-530cff344a44 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:49:18.101033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45dcdc-6c64-4fb9-b591-530cff344a44 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:49:18.104653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac40bb9-507d-488a-8877-82267acfea89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:49:18.105296 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac40bb9-507d-488a-8877-82267acfea89 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:49:18.105584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac40bb9-507d-488a-8877-82267acfea89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:49:18.105864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac40bb9-507d-488a-8877-82267acfea89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:49:18.114522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac40bb9-507d-488a-8877-82267acfea89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:49:18.115954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac40bb9-507d-488a-8877-82267acfea89 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:49:18.125205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1149/2296] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:49:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:49:24.477649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=24 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:49:28.120741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=12 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:49:54.094107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff6e7-9395-427a-9903-93b70eabe92a service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:49:54.105327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ff6e7-9395-427a-9903-93b70eabe92a service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:49:54.110804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec8e22-f8ca-4726-9519-8b8724071899 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:49:54.111435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec8e22-f8ca-4726-9519-8b8724071899 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:49:54.111717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec8e22-f8ca-4726-9519-8b8724071899 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:49:54.111993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec8e22-f8ca-4726-9519-8b8724071899 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:49:54.120518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec8e22-f8ca-4726-9519-8b8724071899 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:49:54.122088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec8e22-f8ca-4726-9519-8b8724071899 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:49:54.131766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1148/2297] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:49:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:50:04.126971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:50:10.471627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac40bb9-507d-488a-8877-82267acfea89 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:50:10.473088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bac40bb9-507d-488a-8877-82267acfea89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:50:10.485063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac40bb9-507d-488a-8877-82267acfea89 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:50:10.488748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930102f1-181a-4b37-9c6d-972d8bf9a1b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:50:10.489394 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930102f1-181a-4b37-9c6d-972d8bf9a1b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:50:10.489794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930102f1-181a-4b37-9c6d-972d8bf9a1b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:50:10.490080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930102f1-181a-4b37-9c6d-972d8bf9a1b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:50:10.498550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930102f1-181a-4b37-9c6d-972d8bf9a1b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:50:10.500005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930102f1-181a-4b37-9c6d-972d8bf9a1b2 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:50:10.508450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1150/2298] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:50:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:50:20.504422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:52:10.275881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec8e22-f8ca-4726-9519-8b8724071899 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:52:10.277412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-40ec8e22-f8ca-4726-9519-8b8724071899 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:52:10.319963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec8e22-f8ca-4726-9519-8b8724071899 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:52:10.323570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a2e2b-035d-4839-93d2-4cd918a759fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:52:10.324211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a2e2b-035d-4839-93d2-4cd918a759fc service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:52:10.324597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a2e2b-035d-4839-93d2-4cd918a759fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:52:10.324874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a2e2b-035d-4839-93d2-4cd918a759fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:52:10.326718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eb8a2e2b-035d-4839-93d2-4cd918a759fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:52:10.422196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a2e2b-035d-4839-93d2-4cd918a759fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:52:10.423633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a2e2b-035d-4839-93d2-4cd918a759fc service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:52:10.444398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1149/2299] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:52:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:52:20.441627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=60 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:53:10.354949 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930102f1-181a-4b37-9c6d-972d8bf9a1b2 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:53:10.373441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930102f1-181a-4b37-9c6d-972d8bf9a1b2 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:53:10.377082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2011962-5cc6-4f74-850a-41f112fef166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:53:10.377712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2011962-5cc6-4f74-850a-41f112fef166 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:53:10.377994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2011962-5cc6-4f74-850a-41f112fef166 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:53:10.378273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2011962-5cc6-4f74-850a-41f112fef166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:53:10.457221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2011962-5cc6-4f74-850a-41f112fef166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:53:10.458602 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2011962-5cc6-4f74-850a-41f112fef166 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:53:10.480141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1151/2300] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:53:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:53:17.751163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a2e2b-035d-4839-93d2-4cd918a759fc service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:53:17.827247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a2e2b-035d-4839-93d2-4cd918a759fc service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:53:17.832459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9128ff2-b368-4386-84f7-465e41eb554c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:53:17.833110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9128ff2-b368-4386-84f7-465e41eb554c service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:53:17.833393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9128ff2-b368-4386-84f7-465e41eb554c service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:53:17.833671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9128ff2-b368-4386-84f7-465e41eb554c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:53:17.835602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a9128ff2-b368-4386-84f7-465e41eb554c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:53:17.859765 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9128ff2-b368-4386-84f7-465e41eb554c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:53:17.861376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9128ff2-b368-4386-84f7-465e41eb554c service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:53:17.871182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1150/2301] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:53:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:53:20.477392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=49 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:53:27.866932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=41 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:53:49.318220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2011962-5cc6-4f74-850a-41f112fef166 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:53:49.327031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2011962-5cc6-4f74-850a-41f112fef166 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:53:49.330653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764ca7b-0fd9-4e6a-8265-7ae6193672a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:53:49.331283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764ca7b-0fd9-4e6a-8265-7ae6193672a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:53:49.331568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764ca7b-0fd9-4e6a-8265-7ae6193672a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:53:49.331846 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764ca7b-0fd9-4e6a-8265-7ae6193672a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:53:49.342673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764ca7b-0fd9-4e6a-8265-7ae6193672a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:53:49.344101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764ca7b-0fd9-4e6a-8265-7ae6193672a1 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:53:49.355976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1152/2302] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:53:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:53:59.350091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:54:05.985615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9128ff2-b368-4386-84f7-465e41eb554c service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:05.995742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9128ff2-b368-4386-84f7-465e41eb554c service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:05.999324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fedb05a-784a-4520-b7cb-a82bf7229835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:05.999957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedb05a-784a-4520-b7cb-a82bf7229835 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:06.000237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedb05a-784a-4520-b7cb-a82bf7229835 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:06.000541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedb05a-784a-4520-b7cb-a82bf7229835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:06.009848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedb05a-784a-4520-b7cb-a82bf7229835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:06.011271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedb05a-784a-4520-b7cb-a82bf7229835 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:06.020232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1151/2303] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:54:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:54:11.522435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764ca7b-0fd9-4e6a-8265-7ae6193672a1 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:11.524032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9764ca7b-0fd9-4e6a-8265-7ae6193672a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:11.619430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764ca7b-0fd9-4e6a-8265-7ae6193672a1 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:11.623981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:11.625102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:11.625398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:11.625675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] PATH_INFO: `/v3/users/e34d1c2cd15b47b39e8273587adc205f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:11.627057 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34d1c2cd15b47b39e8273587adc205f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:11.629249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:11.679524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34d1c2cd15b47b39e8273587adc205f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:54:11.687245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34d1c2cd15b47b39e8273587adc205f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:54:11.687927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34d1c2cd15b47b39e8273587adc205f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:54:11.700637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34d1c2cd15b47b39e8273587adc205f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:54:11.708399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34d1c2cd15b47b39e8273587adc205f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:54:11.714479 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1153/2304] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:54:11 2022] DELETE /identity/v3/users/e34d1c2cd15b47b39e8273587adc205f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:54:11.718970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafbbe1-291c-44d7-8baa-988dd2527cf3 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:11.719505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafbbe1-291c-44d7-8baa-988dd2527cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:11.719629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafbbe1-291c-44d7-8baa-988dd2527cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:11.745510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0fafbbe1-291c-44d7-8baa-988dd2527cf3 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:54:11.746703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0fafbbe1-291c-44d7-8baa-988dd2527cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:11.804945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1152/2305] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:54:11.815785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:11.817220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:11.873630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdf1f9-3a2e-4beb-8d3c-b76e3a65c957 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:11.877177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8704c-a847-46cb-a872-10a6fbf8a4e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:11.877842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8704c-a847-46cb-a872-10a6fbf8a4e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:11.878121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8704c-a847-46cb-a872-10a6fbf8a4e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:11.878393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8704c-a847-46cb-a872-10a6fbf8a4e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:11.884774 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:54:11.889619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8704c-a847-46cb-a872-10a6fbf8a4e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:11.892026 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8704c-a847-46cb-a872-10a6fbf8a4e9 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:11.901885 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1154/2306] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 17:54:12.173413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafbbe1-291c-44d7-8baa-988dd2527cf3 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:12.175565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0fafbbe1-291c-44d7-8baa-988dd2527cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:12.191122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafbbe1-291c-44d7-8baa-988dd2527cf3 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:12.197874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:12.199559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.200206 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.200536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] PATH_INFO: `/v3/projects/66ee4df184ae4eb6a9a559fd40a7ecc1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.201755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=66ee4df184ae4eb6a9a559fd40a7ecc1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:12.204866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:12.264338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66ee4df184ae4eb6a9a559fd40a7ecc1', 'request_id': 'req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:54:12.272538 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66ee4df184ae4eb6a9a559fd40a7ecc1', 'request_id': 'req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:54:12.280624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66ee4df184ae4eb6a9a559fd40a7ecc1', 'request_id': 'req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:54:12.292249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] The token cache is being invalidate because project 66ee4df184ae4eb6a9a559fd40a7ecc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:54:12.292573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:54:12.293221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ed7cdd40-5239-4b81-bb57-2a86c1018c0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:54:12.294835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1153/2307] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:54:12 2022] DELETE /identity/v3/projects/66ee4df184ae4eb6a9a559fd40a7ecc1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:54:12.300574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3507f-2644-4b51-8ecf-ad3f6c848ad9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.300858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3507f-2644-4b51-8ecf-ad3f6c848ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.301134 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3507f-2644-4b51-8ecf-ad3f6c848ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.357676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9ec3507f-2644-4b51-8ecf-ad3f6c848ad9 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:54:12.358884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9ec3507f-2644-4b51-8ecf-ad3f6c848ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:12.422627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1155/2308] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:54:12.430265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12347a-0d3d-48f9-8ad9-af5e238f407d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.430698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12347a-0d3d-48f9-8ad9-af5e238f407d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.431016 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12347a-0d3d-48f9-8ad9-af5e238f407d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.449410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ec12347a-0d3d-48f9-8ad9-af5e238f407d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:54:12.450623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ec12347a-0d3d-48f9-8ad9-af5e238f407d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:12.458543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1154/2309] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:54:12.463370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3507f-2644-4b51-8ecf-ad3f6c848ad9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:12.473129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3507f-2644-4b51-8ecf-ad3f6c848ad9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:12.478422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1acafb-fb70-4fe3-bef8-1ed211f3ca7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:12.479101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1acafb-fb70-4fe3-bef8-1ed211f3ca7d admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.479385 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1acafb-fb70-4fe3-bef8-1ed211f3ca7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.479663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1acafb-fb70-4fe3-bef8-1ed211f3ca7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.480424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1acafb-fb70-4fe3-bef8-1ed211f3ca7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:54:12.480737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1acafb-fb70-4fe3-bef8-1ed211f3ca7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:12.482328 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1acafb-fb70-4fe3-bef8-1ed211f3ca7d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:12.493820 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1156/2310] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 17:54:12.497974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12347a-0d3d-48f9-8ad9-af5e238f407d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:12.507498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12347a-0d3d-48f9-8ad9-af5e238f407d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:12.513645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671c771-7709-46a9-9682-50b58797a22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:12.514674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671c771-7709-46a9-9682-50b58797a22d admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.515031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671c771-7709-46a9-9682-50b58797a22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.515457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671c771-7709-46a9-9682-50b58797a22d admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.516572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671c771-7709-46a9-9682-50b58797a22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:12.519563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671c771-7709-46a9-9682-50b58797a22d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:12.561882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1155/2311] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:54:12 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:54:12.566369 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1acafb-fb70-4fe3-bef8-1ed211f3ca7d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:12.576783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1acafb-fb70-4fe3-bef8-1ed211f3ca7d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:12.634358 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c17da1-a07b-4db0-a4ba-caaa06af55a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:12.635030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c17da1-a07b-4db0-a4ba-caaa06af55a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.635317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c17da1-a07b-4db0-a4ba-caaa06af55a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.635597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c17da1-a07b-4db0-a4ba-caaa06af55a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.636282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c17da1-a07b-4db0-a4ba-caaa06af55a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:12.638231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c17da1-a07b-4db0-a4ba-caaa06af55a5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:12.658532 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1157/2312] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:54:12 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:54:12.662629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671c771-7709-46a9-9682-50b58797a22d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:12.673297 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671c771-7709-46a9-9682-50b58797a22d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:12.678084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5318-952f-4c7d-b034-c045b9db723e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:12.678751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5318-952f-4c7d-b034-c045b9db723e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.679037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5318-952f-4c7d-b034-c045b9db723e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.679316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5318-952f-4c7d-b034-c045b9db723e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.680046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce5318-952f-4c7d-b034-c045b9db723e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:12.682432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce5318-952f-4c7d-b034-c045b9db723e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:12.692708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1156/2313] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:54:12.697171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c17da1-a07b-4db0-a4ba-caaa06af55a5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:12.707028 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c17da1-a07b-4db0-a4ba-caaa06af55a5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:12.711518 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbeb90-2553-4c5e-8b06-bd861e0bd94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:12.712398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbeb90-2553-4c5e-8b06-bd861e0bd94b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.712684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbeb90-2553-4c5e-8b06-bd861e0bd94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.712961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbeb90-2553-4c5e-8b06-bd861e0bd94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.713681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbeb90-2553-4c5e-8b06-bd861e0bd94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:12.715267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbeb90-2553-4c5e-8b06-bd861e0bd94b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:12.725075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1158/2314] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:54:12.729321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5318-952f-4c7d-b034-c045b9db723e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:12.739100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5318-952f-4c7d-b034-c045b9db723e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:12.744344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a3df6-a3a4-4a38-8b47-967fb60b767b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:12.745221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3df6-a3a4-4a38-8b47-967fb60b767b admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.745505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3df6-a3a4-4a38-8b47-967fb60b767b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.745788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3df6-a3a4-4a38-8b47-967fb60b767b admin admin] PATH_INFO: `/v3/projects/998f0fe6849446b48efc5c1b4a17babf/users/dc42172de85a4e4999bcfe8ceac87ee7/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.757583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:54:12.761613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a3df6-a3a4-4a38-8b47-967fb60b767b admin admin] RBAC: Authorizing `identity:create_grant(project_id=998f0fe6849446b48efc5c1b4a17babf, user_id=dc42172de85a4e4999bcfe8ceac87ee7, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:12.765327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a3df6-a3a4-4a38-8b47-967fb60b767b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:12.778676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1157/2315] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:54:12 2022] PUT /identity/v3/projects/998f0fe6849446b48efc5c1b4a17babf/users/dc42172de85a4e4999bcfe8ceac87ee7/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:54:12.789625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29849427-fc2f-4d12-ac93-bcfda0c04fdf None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.790500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29849427-fc2f-4d12-ac93-bcfda0c04fdf None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.790980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29849427-fc2f-4d12-ac93-bcfda0c04fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.817242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-29849427-fc2f-4d12-ac93-bcfda0c04fdf None None] MFA Rules not processed for user `dc42172de85a4e4999bcfe8ceac87ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:54:12.818495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-29849427-fc2f-4d12-ac93-bcfda0c04fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:12.841541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:54:12.884879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1159/2316] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:54:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:54:12.895248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a3df6-a3a4-4a38-8b47-967fb60b767b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:12.896932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-276a3df6-a3a4-4a38-8b47-967fb60b767b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:12.973638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a3df6-a3a4-4a38-8b47-967fb60b767b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:12.977875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f5e73-0c6f-4de4-bbe5-21720ac29d1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:12.978636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f5e73-0c6f-4de4-bbe5-21720ac29d1c service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:12.978924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f5e73-0c6f-4de4-bbe5-21720ac29d1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:12.979210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f5e73-0c6f-4de4-bbe5-21720ac29d1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:12.987419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f5e73-0c6f-4de4-bbe5-21720ac29d1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:12.989077 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f5e73-0c6f-4de4-bbe5-21720ac29d1c service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:12.998689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1158/2317] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:54:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:54:14.379381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29849427-fc2f-4d12-ac93-bcfda0c04fdf None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:14.389348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29849427-fc2f-4d12-ac93-bcfda0c04fdf None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:14.392903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363df30c-c69f-4df1-9e96-e5ab812c124b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:14.393539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363df30c-c69f-4df1-9e96-e5ab812c124b service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:14.393824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363df30c-c69f-4df1-9e96-e5ab812c124b service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:14.394097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363df30c-c69f-4df1-9e96-e5ab812c124b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:14.395825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-363df30c-c69f-4df1-9e96-e5ab812c124b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:14.453623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363df30c-c69f-4df1-9e96-e5ab812c124b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:14.455108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363df30c-c69f-4df1-9e96-e5ab812c124b service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:14.464188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1160/2318] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:54:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:54:18.764086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f5e73-0c6f-4de4-bbe5-21720ac29d1c service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:18.772688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f5e73-0c6f-4de4-bbe5-21720ac29d1c service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:18.777161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7a0d5-9bd3-4ea3-9e87-7bf3b802511b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:18.777800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7a0d5-9bd3-4ea3-9e87-7bf3b802511b service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:18.778080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7a0d5-9bd3-4ea3-9e87-7bf3b802511b service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:18.778354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7a0d5-9bd3-4ea3-9e87-7bf3b802511b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:18.780108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e7e7a0d5-9bd3-4ea3-9e87-7bf3b802511b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:18.839055 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7a0d5-9bd3-4ea3-9e87-7bf3b802511b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:18.840540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7a0d5-9bd3-4ea3-9e87-7bf3b802511b service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:18.849523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1159/2319] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:54:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:54:24.459858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=44 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:54:28.845329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=66,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:54:53.518392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363df30c-c69f-4df1-9e96-e5ab812c124b service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:54:53.528142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363df30c-c69f-4df1-9e96-e5ab812c124b service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:54:53.531820 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bc35c-19fc-4432-84a5-219a14f51155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:54:53.532508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bc35c-19fc-4432-84a5-219a14f51155 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:54:53.532788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bc35c-19fc-4432-84a5-219a14f51155 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:54:53.533064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bc35c-19fc-4432-84a5-219a14f51155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:54:53.534842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-669bc35c-19fc-4432-84a5-219a14f51155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:54:53.595104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bc35c-19fc-4432-84a5-219a14f51155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:54:53.596572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bc35c-19fc-4432-84a5-219a14f51155 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:54:53.605698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1161/2320] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:54:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:03.601261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=28 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:55:06.496940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7a0d5-9bd3-4ea3-9e87-7bf3b802511b service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:06.498287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e7e7a0d5-9bd3-4ea3-9e87-7bf3b802511b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:06.605369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7a0d5-9bd3-4ea3-9e87-7bf3b802511b service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:06.610934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc1157-f840-4bd2-8b56-6e228d724257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:06.611594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc1157-f840-4bd2-8b56-6e228d724257 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:06.611881 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc1157-f840-4bd2-8b56-6e228d724257 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:06.612160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc1157-f840-4bd2-8b56-6e228d724257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:06.613987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b4cc1157-f840-4bd2-8b56-6e228d724257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:06.626824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc1157-f840-4bd2-8b56-6e228d724257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:06.628464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc1157-f840-4bd2-8b56-6e228d724257 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:06.639695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1160/2321] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:55:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:09.943030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bc35c-19fc-4432-84a5-219a14f51155 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:09.944494 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-669bc35c-19fc-4432-84a5-219a14f51155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:10.022568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bc35c-19fc-4432-84a5-219a14f51155 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:10.026227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e54a5-384d-40c9-99a4-ef0ed155c550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:10.026866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e54a5-384d-40c9-99a4-ef0ed155c550 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:10.027173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e54a5-384d-40c9-99a4-ef0ed155c550 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:10.027491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e54a5-384d-40c9-99a4-ef0ed155c550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:10.029679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f41e54a5-384d-40c9-99a4-ef0ed155c550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:10.092900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e54a5-384d-40c9-99a4-ef0ed155c550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:10.094292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e54a5-384d-40c9-99a4-ef0ed155c550 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:10.104072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1162/2322] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:55:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:13.390911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc1157-f840-4bd2-8b56-6e228d724257 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:13.392491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b4cc1157-f840-4bd2-8b56-6e228d724257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:13.450864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e54a5-384d-40c9-99a4-ef0ed155c550 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:13.452898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f41e54a5-384d-40c9-99a4-ef0ed155c550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:13.474223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e54a5-384d-40c9-99a4-ef0ed155c550 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:13.475568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc1157-f840-4bd2-8b56-6e228d724257 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:13.481285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:13.481735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964a925-796f-4978-a5b2-d5f31f03daaf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:13.482318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:13.482454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a925-796f-4978-a5b2-d5f31f03daaf service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:13.482628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:13.482838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a925-796f-4978-a5b2-d5f31f03daaf service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:13.483122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a925-796f-4978-a5b2-d5f31f03daaf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:13.483440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] PATH_INFO: `/v3/users/1bf6e7a4afaf405caae7d750973d9956` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:13.484954 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a964a925-796f-4978-a5b2-d5f31f03daaf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:13.501854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf6e7a4afaf405caae7d750973d9956)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:13.503669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964a925-796f-4978-a5b2-d5f31f03daaf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:13.504662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:13.505300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964a925-796f-4978-a5b2-d5f31f03daaf service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:13.516773 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1161/2323] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:55:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:13.559966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf6e7a4afaf405caae7d750973d9956'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:55:13.569731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf6e7a4afaf405caae7d750973d9956'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:55:13.570418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf6e7a4afaf405caae7d750973d9956'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:55:13.582094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf6e7a4afaf405caae7d750973d9956'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:55:13.583792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:55:13.590427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f72c9e85-c2db-4fdc-8ae1-b246ed47f826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf6e7a4afaf405caae7d750973d9956'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:55:13.596760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1163/2324] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 17:55:13 2022] DELETE /identity/v3/users/1bf6e7a4afaf405caae7d750973d9956 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:55:13.776616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964a925-796f-4978-a5b2-d5f31f03daaf service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:13.791384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964a925-796f-4978-a5b2-d5f31f03daaf service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:13.850231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:13.851120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:13.851400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:13.851672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] PATH_INFO: `/v3/projects/21c11b3abe754d86919bbcf31ecb312a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:13.858927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:55:13.863433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c11b3abe754d86919bbcf31ecb312a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:13.865381 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:13.910909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c11b3abe754d86919bbcf31ecb312a', 'request_id': 'req-d2ea76e4-ee0c-4f76-b72b-841faeefb128'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:55:13.919407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c11b3abe754d86919bbcf31ecb312a', 'request_id': 'req-d2ea76e4-ee0c-4f76-b72b-841faeefb128'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:55:13.925860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c11b3abe754d86919bbcf31ecb312a', 'request_id': 'req-d2ea76e4-ee0c-4f76-b72b-841faeefb128'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:55:13.938704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] The token cache is being invalidate because project 21c11b3abe754d86919bbcf31ecb312a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 17:55:13.938994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:55:13.939646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d2ea76e4-ee0c-4f76-b72b-841faeefb128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 17:55:13.941473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1162/2325] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 17:55:13 2022] DELETE /identity/v3/projects/21c11b3abe754d86919bbcf31ecb312a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:55:13.947244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c6896-c939-40a5-86d0-057791575283 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:13.947536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c6896-c939-40a5-86d0-057791575283 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:13.947814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c6896-c939-40a5-86d0-057791575283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:13.967054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d13c6896-c939-40a5-86d0-057791575283 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:55:13.968194 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d13c6896-c939-40a5-86d0-057791575283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:14.028253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1164/2326] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:55:14.036080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41d3db-4668-49b7-a69d-dafbf00cc982 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:14.036287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41d3db-4668-49b7-a69d-dafbf00cc982 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:14.036595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41d3db-4668-49b7-a69d-dafbf00cc982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:14.056465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-9a41d3db-4668-49b7-a69d-dafbf00cc982 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:55:14.057670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9a41d3db-4668-49b7-a69d-dafbf00cc982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:14.066301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1163/2327] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:55:14.070529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c6896-c939-40a5-86d0-057791575283 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:14.080619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c6896-c939-40a5-86d0-057791575283 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:14.085436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af1e8b-7643-4aad-944b-14f61394bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:14.086106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af1e8b-7643-4aad-944b-14f61394bf0c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:14.086392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af1e8b-7643-4aad-944b-14f61394bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:14.086670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af1e8b-7643-4aad-944b-14f61394bf0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:14.087586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af1e8b-7643-4aad-944b-14f61394bf0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 17:55:14.087899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af1e8b-7643-4aad-944b-14f61394bf0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:14.089768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af1e8b-7643-4aad-944b-14f61394bf0c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:14.103009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1165/2328] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 17:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 17:55:14.108309 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41d3db-4668-49b7-a69d-dafbf00cc982 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:14.117808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41d3db-4668-49b7-a69d-dafbf00cc982 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:14.126119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e115d1-7550-4ee8-a983-43e30322fee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:14.126119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e115d1-7550-4ee8-a983-43e30322fee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:14.126119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e115d1-7550-4ee8-a983-43e30322fee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:14.126119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e115d1-7550-4ee8-a983-43e30322fee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:14.126451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e115d1-7550-4ee8-a983-43e30322fee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:14.127078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e115d1-7550-4ee8-a983-43e30322fee5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:14.157487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1164/2329] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 17:55:14 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:55:14.161532 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af1e8b-7643-4aad-944b-14f61394bf0c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:14.171808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af1e8b-7643-4aad-944b-14f61394bf0c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:14.226748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b8eea-47e7-4d18-b2f1-ce927ae23f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:14.227416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b8eea-47e7-4d18-b2f1-ce927ae23f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:14.227698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b8eea-47e7-4d18-b2f1-ce927ae23f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:14.227974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b8eea-47e7-4d18-b2f1-ce927ae23f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:14.228723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b8eea-47e7-4d18-b2f1-ce927ae23f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:14.230744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b8eea-47e7-4d18-b2f1-ce927ae23f5c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:14.249947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1166/2330] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 17:55:14 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 17:55:14.254064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e115d1-7550-4ee8-a983-43e30322fee5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:14.265281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e115d1-7550-4ee8-a983-43e30322fee5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:14.269854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff4046-51e5-458e-9a7f-08ab2d998778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:14.270514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff4046-51e5-458e-9a7f-08ab2d998778 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:14.270797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff4046-51e5-458e-9a7f-08ab2d998778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:14.271071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff4046-51e5-458e-9a7f-08ab2d998778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:14.271794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff4046-51e5-458e-9a7f-08ab2d998778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:14.273415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff4046-51e5-458e-9a7f-08ab2d998778 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:14.285033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1165/2331] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:55:14.289668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b8eea-47e7-4d18-b2f1-ce927ae23f5c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:14.300173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b8eea-47e7-4d18-b2f1-ce927ae23f5c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:14.305001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575c0d8-737f-4c72-87f6-e33dfb61f558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:14.305671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575c0d8-737f-4c72-87f6-e33dfb61f558 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:14.305956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575c0d8-737f-4c72-87f6-e33dfb61f558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:14.306349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575c0d8-737f-4c72-87f6-e33dfb61f558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:14.307094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575c0d8-737f-4c72-87f6-e33dfb61f558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:14.308752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575c0d8-737f-4c72-87f6-e33dfb61f558 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:14.319790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1167/2332] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 17:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 17:55:14.323930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff4046-51e5-458e-9a7f-08ab2d998778 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:14.334386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff4046-51e5-458e-9a7f-08ab2d998778 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:14.338865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1048f49-b111-46df-828d-11036a9265e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:14.339732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048f49-b111-46df-828d-11036a9265e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:14.340767 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048f49-b111-46df-828d-11036a9265e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:14.341052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048f49-b111-46df-828d-11036a9265e3 admin admin] PATH_INFO: `/v3/projects/444abcf11d564f7692af245f2bd98a84/users/d08a5eaa6d814e388b0af0581a530888/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:14.355605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048f49-b111-46df-828d-11036a9265e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444abcf11d564f7692af245f2bd98a84, user_id=d08a5eaa6d814e388b0af0581a530888, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:14.358298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048f49-b111-46df-828d-11036a9265e3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:14.370921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1166/2333] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 17:55:14 2022] PUT /identity/v3/projects/444abcf11d564f7692af245f2bd98a84/users/d08a5eaa6d814e388b0af0581a530888/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 17:55:14.375995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c148b-75af-4254-b2f9-24f27b8a516a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:14.376315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c148b-75af-4254-b2f9-24f27b8a516a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:14.376599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c148b-75af-4254-b2f9-24f27b8a516a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:14.394772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ec9c148b-75af-4254-b2f9-24f27b8a516a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:55:14.395990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ec9c148b-75af-4254-b2f9-24f27b8a516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:14.446335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:55:14.457909 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1168/2334] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:55:14.468103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1048f49-b111-46df-828d-11036a9265e3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:14.469989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c1048f49-b111-46df-828d-11036a9265e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:14.527465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1048f49-b111-46df-828d-11036a9265e3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:14.531107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2347e-ddf9-4de2-94df-dec350a32a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:14.531805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2347e-ddf9-4de2-94df-dec350a32a03 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:14.532036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2347e-ddf9-4de2-94df-dec350a32a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:14.532331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2347e-ddf9-4de2-94df-dec350a32a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:14.543798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2347e-ddf9-4de2-94df-dec350a32a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:14.545256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2347e-ddf9-4de2-94df-dec350a32a03 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:14.555149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1167/2335] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:55:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:15.473679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c148b-75af-4254-b2f9-24f27b8a516a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:15.475125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ec9c148b-75af-4254-b2f9-24f27b8a516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:15.488030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c148b-75af-4254-b2f9-24f27b8a516a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:15.493556 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c1939-b6d2-47f8-88c5-781d5c228fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:15.494227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c1939-b6d2-47f8-88c5-781d5c228fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:15.494512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c1939-b6d2-47f8-88c5-781d5c228fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:15.494789 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c1939-b6d2-47f8-88c5-781d5c228fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:15.498608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-726c1939-b6d2-47f8-88c5-781d5c228fc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:15.509832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c1939-b6d2-47f8-88c5-781d5c228fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:15.512919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c1939-b6d2-47f8-88c5-781d5c228fc0 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:15.522667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1169/2336] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:55:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:19.021575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dc689-c374-44e5-93c7-66cb47eb36eb None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:19.022138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dc689-c374-44e5-93c7-66cb47eb36eb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:19.022266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dc689-c374-44e5-93c7-66cb47eb36eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:19.052106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f18dc689-c374-44e5-93c7-66cb47eb36eb None None] MFA Rules not processed for user `d08a5eaa6d814e388b0af0581a530888`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:55:19.053379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f18dc689-c374-44e5-93c7-66cb47eb36eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:19.068526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:55:19.129542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1168/2337] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:55:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:55:19.144505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c1939-b6d2-47f8-88c5-781d5c228fc0 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:19.146973 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-726c1939-b6d2-47f8-88c5-781d5c228fc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:19.227962 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c1939-b6d2-47f8-88c5-781d5c228fc0 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:19.233981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dc614-479f-4305-9649-94bbc1180334 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:19.234637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc614-479f-4305-9649-94bbc1180334 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:19.234921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc614-479f-4305-9649-94bbc1180334 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:19.235322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc614-479f-4305-9649-94bbc1180334 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:19.244198 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dc614-479f-4305-9649-94bbc1180334 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:19.245833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dc614-479f-4305-9649-94bbc1180334 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:19.256154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1170/2338] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:55:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:20.492366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3355ca2-b9de-4fa1-8091-ed208897c0ba None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:20.492867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3355ca2-b9de-4fa1-8091-ed208897c0ba None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:20.498892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3355ca2-b9de-4fa1-8091-ed208897c0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:20.518341 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c3355ca2-b9de-4fa1-8091-ed208897c0ba None None] MFA Rules not processed for user `d08a5eaa6d814e388b0af0581a530888`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 17:55:20.519629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c3355ca2-b9de-4fa1-8091-ed208897c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:20.526680 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1169/2339] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 17:55:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 17:55:20.861218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dc614-479f-4305-9649-94bbc1180334 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:20.870838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dc614-479f-4305-9649-94bbc1180334 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:20.878899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0c546-c145-40c4-b00c-ca5d2752051d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:20.879913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0c546-c145-40c4-b00c-ca5d2752051d service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:20.881540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0c546-c145-40c4-b00c-ca5d2752051d service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:20.881540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0c546-c145-40c4-b00c-ca5d2752051d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:20.883553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f4d0c546-c145-40c4-b00c-ca5d2752051d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:20.896876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0c546-c145-40c4-b00c-ca5d2752051d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:20.898299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0c546-c145-40c4-b00c-ca5d2752051d service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:20.909617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1171/2340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:55:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:22.326497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3355ca2-b9de-4fa1-8091-ed208897c0ba None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:22.328165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c3355ca2-b9de-4fa1-8091-ed208897c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:22.420056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3355ca2-b9de-4fa1-8091-ed208897c0ba None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:22.425203 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f1e65-5531-48dd-adfd-7f92337d98f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:22.425865 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f1e65-5531-48dd-adfd-7f92337d98f9 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:22.426151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f1e65-5531-48dd-adfd-7f92337d98f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:22.426429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f1e65-5531-48dd-adfd-7f92337d98f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:22.428225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-081f1e65-5531-48dd-adfd-7f92337d98f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:22.442168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f1e65-5531-48dd-adfd-7f92337d98f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:22.443766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f1e65-5531-48dd-adfd-7f92337d98f9 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:22.455517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1170/2341] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:55:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:30.905277 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=47 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:55:32.451453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=56 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:55:34.393899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0c546-c145-40c4-b00c-ca5d2752051d service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:34.409906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0c546-c145-40c4-b00c-ca5d2752051d service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:34.415280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d12387-1e86-4244-b96d-b1adb56b14be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:34.415919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d12387-1e86-4244-b96d-b1adb56b14be service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:34.416201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d12387-1e86-4244-b96d-b1adb56b14be service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:34.416507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d12387-1e86-4244-b96d-b1adb56b14be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:34.418305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a9d12387-1e86-4244-b96d-b1adb56b14be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:34.430681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d12387-1e86-4244-b96d-b1adb56b14be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:34.432318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d12387-1e86-4244-b96d-b1adb56b14be service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:34.443512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1172/2342] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:55:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:37.641721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f1e65-5531-48dd-adfd-7f92337d98f9 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:55:37.654610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f1e65-5531-48dd-adfd-7f92337d98f9 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:55:37.660158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6547ef8-8fe7-496c-b993-c01331305254 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:55:37.660903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6547ef8-8fe7-496c-b993-c01331305254 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:55:37.661254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6547ef8-8fe7-496c-b993-c01331305254 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:55:37.661600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6547ef8-8fe7-496c-b993-c01331305254 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:55:37.663535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d6547ef8-8fe7-496c-b993-c01331305254 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:55:37.676036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6547ef8-8fe7-496c-b993-c01331305254 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:55:37.677647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6547ef8-8fe7-496c-b993-c01331305254 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:55:37.688722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1171/2343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:55:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:55:44.438719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:55:47.684028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:58:10.415647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d12387-1e86-4244-b96d-b1adb56b14be service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:58:10.425140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d12387-1e86-4244-b96d-b1adb56b14be service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:58:10.428790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8cf92-bccd-4eeb-880d-13f38624f41e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:58:10.429425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8cf92-bccd-4eeb-880d-13f38624f41e service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:58:10.429705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8cf92-bccd-4eeb-880d-13f38624f41e service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:58:10.429978 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8cf92-bccd-4eeb-880d-13f38624f41e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:58:10.431710 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-47e8cf92-bccd-4eeb-880d-13f38624f41e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:58:10.444957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8cf92-bccd-4eeb-880d-13f38624f41e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:58:10.446401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8cf92-bccd-4eeb-880d-13f38624f41e service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:58:10.456872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1173/2344] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:58:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:58:20.452275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:58:24.739818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6547ef8-8fe7-496c-b993-c01331305254 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:58:24.741458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d6547ef8-8fe7-496c-b993-c01331305254 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:58:24.755840 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6547ef8-8fe7-496c-b993-c01331305254 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:58:24.761281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cbcb5-754e-4599-9ef7-fea996e0e8f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:58:24.761928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cbcb5-754e-4599-9ef7-fea996e0e8f3 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:58:24.762212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cbcb5-754e-4599-9ef7-fea996e0e8f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:58:24.762491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cbcb5-754e-4599-9ef7-fea996e0e8f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:58:24.764244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-615cbcb5-754e-4599-9ef7-fea996e0e8f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:58:24.776490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615cbcb5-754e-4599-9ef7-fea996e0e8f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:58:24.778050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615cbcb5-754e-4599-9ef7-fea996e0e8f3 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:58:24.788273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1172/2345] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:58:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:58:34.783549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:58:51.054901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8cf92-bccd-4eeb-880d-13f38624f41e service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:58:51.063824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8cf92-bccd-4eeb-880d-13f38624f41e service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:58:51.068097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0ac88-06ad-4c03-a43e-705183a1236e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:58:51.068769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0ac88-06ad-4c03-a43e-705183a1236e service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:58:51.069192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0ac88-06ad-4c03-a43e-705183a1236e service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:58:51.069595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0ac88-06ad-4c03-a43e-705183a1236e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:58:51.071439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-16e0ac88-06ad-4c03-a43e-705183a1236e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:58:51.085001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0ac88-06ad-4c03-a43e-705183a1236e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:58:51.086435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0ac88-06ad-4c03-a43e-705183a1236e service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:58:51.097154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1174/2346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:58:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:59:01.092440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:59:15.244542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cbcb5-754e-4599-9ef7-fea996e0e8f3 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:59:15.245946 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-615cbcb5-754e-4599-9ef7-fea996e0e8f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:59:15.261597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cbcb5-754e-4599-9ef7-fea996e0e8f3 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:59:15.266990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13fecd-586c-4948-903b-14e42900a046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:59:15.267623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13fecd-586c-4948-903b-14e42900a046 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:59:15.267904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13fecd-586c-4948-903b-14e42900a046 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:59:15.268185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13fecd-586c-4948-903b-14e42900a046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:59:15.269912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4b13fecd-586c-4948-903b-14e42900a046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:59:15.348535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13fecd-586c-4948-903b-14e42900a046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:59:15.350140 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13fecd-586c-4948-903b-14e42900a046 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:59:15.368425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1173/2347] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:59:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:59:25.365443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=41 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 17:59:53.758108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0ac88-06ad-4c03-a43e-705183a1236e service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:59:53.760036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-16e0ac88-06ad-4c03-a43e-705183a1236e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:59:53.814604 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0ac88-06ad-4c03-a43e-705183a1236e service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:59:53.818122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c92a3-f060-46f4-a9cf-e113e91f95cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:59:53.818759 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c92a3-f060-46f4-a9cf-e113e91f95cb service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:59:53.819038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c92a3-f060-46f4-a9cf-e113e91f95cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:59:53.819310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c92a3-f060-46f4-a9cf-e113e91f95cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:59:53.821059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d38c92a3-f060-46f4-a9cf-e113e91f95cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:59:53.905930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c92a3-f060-46f4-a9cf-e113e91f95cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:59:53.907370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c92a3-f060-46f4-a9cf-e113e91f95cb service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:59:53.926177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1175/2348] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:59:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:59:53.955222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13fecd-586c-4948-903b-14e42900a046 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:59:53.964970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13fecd-586c-4948-903b-14e42900a046 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:59:53.968473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7cbf9-0580-4d85-85ea-4c08ad4722f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:59:53.969134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7cbf9-0580-4d85-85ea-4c08ad4722f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:59:53.969413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7cbf9-0580-4d85-85ea-4c08ad4722f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:59:53.969688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7cbf9-0580-4d85-85ea-4c08ad4722f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:59:53.971403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b7d7cbf9-0580-4d85-85ea-4c08ad4722f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:59:54.052045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7cbf9-0580-4d85-85ea-4c08ad4722f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:59:54.053531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7cbf9-0580-4d85-85ea-4c08ad4722f1 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:59:54.072650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1174/2349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:59:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 17:59:54.115198 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c92a3-f060-46f4-a9cf-e113e91f95cb service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 17:59:54.124630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c92a3-f060-46f4-a9cf-e113e91f95cb service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 17:59:54.128185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688367-e305-4dad-be24-35997e5277a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 17:59:54.128854 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688367-e305-4dad-be24-35997e5277a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 17:59:54.129135 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688367-e305-4dad-be24-35997e5277a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 17:59:54.129413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688367-e305-4dad-be24-35997e5277a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 17:59:54.131146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8b688367-e305-4dad-be24-35997e5277a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 17:59:54.189465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b688367-e305-4dad-be24-35997e5277a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 17:59:54.190889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b688367-e305-4dad-be24-35997e5277a5 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 17:59:54.200381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1176/2350] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 17:59:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:00:04.069256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=41 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:00:04.195587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:00:22.592988 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7cbf9-0580-4d85-85ea-4c08ad4722f1 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:00:22.602187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7cbf9-0580-4d85-85ea-4c08ad4722f1 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:00:22.605862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32af703-467f-4ad9-b5fc-aa617171b126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:00:22.606492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32af703-467f-4ad9-b5fc-aa617171b126 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:00:22.606771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32af703-467f-4ad9-b5fc-aa617171b126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:00:22.607041 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32af703-467f-4ad9-b5fc-aa617171b126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:00:22.615048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32af703-467f-4ad9-b5fc-aa617171b126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:00:22.616492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32af703-467f-4ad9-b5fc-aa617171b126 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:00:22.625135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1175/2351] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:00:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:00:32.621572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:01:01.387815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688367-e305-4dad-be24-35997e5277a5 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:01:01.389205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8b688367-e305-4dad-be24-35997e5277a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:01:01.401304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688367-e305-4dad-be24-35997e5277a5 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:01:01.405281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be37248-d70b-4a39-a29a-17ba8c04a32c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:01:01.405916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be37248-d70b-4a39-a29a-17ba8c04a32c service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:01:01.406194 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be37248-d70b-4a39-a29a-17ba8c04a32c service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:01:01.406467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be37248-d70b-4a39-a29a-17ba8c04a32c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:01:01.414962 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be37248-d70b-4a39-a29a-17ba8c04a32c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:01:01.417156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be37248-d70b-4a39-a29a-17ba8c04a32c service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:01:01.425784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1177/2352] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:01:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:01:11.421836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:01:53.694914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301780-b278-48c6-82d1-56223cd3254b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:01:53.695709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301780-b278-48c6-82d1-56223cd3254b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:01:53.695709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301780-b278-48c6-82d1-56223cd3254b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:01:53.726456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-95301780-b278-48c6-82d1-56223cd3254b None None] MFA Rules not processed for user `2ea3ea36439143b8a55f181170efc14a`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:01:53.727744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-95301780-b278-48c6-82d1-56223cd3254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:01:53.735057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1176/2353] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:01:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:01:54.035790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be37248-d70b-4a39-a29a-17ba8c04a32c service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:01:54.045548 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be37248-d70b-4a39-a29a-17ba8c04a32c service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:01:54.049075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3388718-47a9-46ea-b13c-fb840c68f44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:01:54.049709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3388718-47a9-46ea-b13c-fb840c68f44e service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:01:54.049988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3388718-47a9-46ea-b13c-fb840c68f44e service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:01:54.050263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3388718-47a9-46ea-b13c-fb840c68f44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:01:54.052021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b3388718-47a9-46ea-b13c-fb840c68f44e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:01:54.062711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3388718-47a9-46ea-b13c-fb840c68f44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:01:54.064133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3388718-47a9-46ea-b13c-fb840c68f44e service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:01:54.072644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1178/2354] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:01:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:01:54.432380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95301780-b278-48c6-82d1-56223cd3254b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:01:54.452424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95301780-b278-48c6-82d1-56223cd3254b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:01:54.457492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c8040-7f49-499c-a33f-6f5d5540205d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:01:54.458115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8040-7f49-499c-a33f-6f5d5540205d service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:01:54.458397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8040-7f49-499c-a33f-6f5d5540205d service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:01:54.458687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8040-7f49-499c-a33f-6f5d5540205d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:01:54.467120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3388718-47a9-46ea-b13c-fb840c68f44e service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:01:54.468724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b3388718-47a9-46ea-b13c-fb840c68f44e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:01:54.475750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c8040-7f49-499c-a33f-6f5d5540205d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:01:54.477612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c8040-7f49-499c-a33f-6f5d5540205d service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:01:54.484840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3388718-47a9-46ea-b13c-fb840c68f44e service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:01:54.489598 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:01:54.490716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:01:54.491084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:01:54.491262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] PATH_INFO: `/v3/users/dc42172de85a4e4999bcfe8ceac87ee7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:01:54.492441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc42172de85a4e4999bcfe8ceac87ee7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:01:54.496124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:01:54.519709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1177/2355] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:01:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:01:54.554446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc42172de85a4e4999bcfe8ceac87ee7'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:01:54.578570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc42172de85a4e4999bcfe8ceac87ee7'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:01:54.579607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc42172de85a4e4999bcfe8ceac87ee7'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:01:54.596741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc42172de85a4e4999bcfe8ceac87ee7'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:01:54.606625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc42172de85a4e4999bcfe8ceac87ee7'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:01:54.617545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1179/2356] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:01:54 2022] DELETE /identity/v3/users/dc42172de85a4e4999bcfe8ceac87ee7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:01:54.623084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e670331-2ae0-42db-8351-5042f97e2763 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:01:54.623287 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e670331-2ae0-42db-8351-5042f97e2763 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:01:54.623494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e670331-2ae0-42db-8351-5042f97e2763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:01:54.645104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0e670331-2ae0-42db-8351-5042f97e2763 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:01:54.646688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0e670331-2ae0-42db-8351-5042f97e2763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:01:54.716280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1178/2357] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:01:54.726829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:01:54.729589 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:01:54.793667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3b24e-0565-4f87-b14a-c289d3e98126 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:01:54.797469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d10728-98ed-4d0c-b283-2c6cbd9cf0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:01:54.798119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d10728-98ed-4d0c-b283-2c6cbd9cf0aa service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:01:54.798397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d10728-98ed-4d0c-b283-2c6cbd9cf0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:01:54.798668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d10728-98ed-4d0c-b283-2c6cbd9cf0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:01:54.809202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d10728-98ed-4d0c-b283-2c6cbd9cf0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:01:54.810934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d10728-98ed-4d0c-b283-2c6cbd9cf0aa service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:01:54.820806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1180/2358] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:01:55.127032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e670331-2ae0-42db-8351-5042f97e2763 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:01:55.129003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0e670331-2ae0-42db-8351-5042f97e2763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:01:55.163816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e670331-2ae0-42db-8351-5042f97e2763 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:01:55.170270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:01:55.171298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:01:55.171798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:01:55.172144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] PATH_INFO: `/v3/projects/998f0fe6849446b48efc5c1b4a17babf` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:01:55.174282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] RBAC: Authorizing `identity:delete_project(project_id=998f0fe6849446b48efc5c1b4a17babf)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:01:55.176391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:01:55.235766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '998f0fe6849446b48efc5c1b4a17babf', 'request_id': 'req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:01:55.269110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '998f0fe6849446b48efc5c1b4a17babf', 'request_id': 'req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:01:55.277012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '998f0fe6849446b48efc5c1b4a17babf', 'request_id': 'req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:01:55.291596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] The token cache is being invalidate because project 998f0fe6849446b48efc5c1b4a17babf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:01:55.291942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:01:55.292986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:01:55.295171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1179/2359] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:01:55 2022] DELETE /identity/v3/projects/998f0fe6849446b48efc5c1b4a17babf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:01:57.088536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d10728-98ed-4d0c-b283-2c6cbd9cf0aa service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:01:57.091127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-29d10728-98ed-4d0c-b283-2c6cbd9cf0aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:01:57.165853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:01:57.187105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d10728-98ed-4d0c-b283-2c6cbd9cf0aa service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:01:57.267409 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede77e07-1df7-4059-91da-ef2364d18365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:01:57.268649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77e07-1df7-4059-91da-ef2364d18365 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:01:57.269006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77e07-1df7-4059-91da-ef2364d18365 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:01:57.269370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77e07-1df7-4059-91da-ef2364d18365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:01:57.272157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ede77e07-1df7-4059-91da-ef2364d18365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:01:57.286051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede77e07-1df7-4059-91da-ef2364d18365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:01:57.287700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede77e07-1df7-4059-91da-ef2364d18365 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:01:57.301659 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1181/2360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:01:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:02:02.324143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:02:02.325647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:02:02.342316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb0b4b-6e66-47ee-925e-3fdc395ff57e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:02:02.349094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94986274-c8bc-48a2-8dba-58ee032c3d22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:02:02.349760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94986274-c8bc-48a2-8dba-58ee032c3d22 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:02:02.350049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94986274-c8bc-48a2-8dba-58ee032c3d22 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:02:02.350333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94986274-c8bc-48a2-8dba-58ee032c3d22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:02:02.352085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-94986274-c8bc-48a2-8dba-58ee032c3d22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:02:02.364990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94986274-c8bc-48a2-8dba-58ee032c3d22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:02:02.366584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94986274-c8bc-48a2-8dba-58ee032c3d22 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:02:02.378204 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1180/2361] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:02:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:02:07.297700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=34 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:02:12.373629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:03:30.715240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede77e07-1df7-4059-91da-ef2364d18365 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:03:30.716733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ede77e07-1df7-4059-91da-ef2364d18365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:03:30.853902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede77e07-1df7-4059-91da-ef2364d18365 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:03:30.859452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09797c-971e-49cf-afea-0cdecd626fc6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:03:30.860094 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09797c-971e-49cf-afea-0cdecd626fc6 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:03:30.860404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09797c-971e-49cf-afea-0cdecd626fc6 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:03:30.860679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09797c-971e-49cf-afea-0cdecd626fc6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:03:30.862416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dd09797c-971e-49cf-afea-0cdecd626fc6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:03:30.873799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09797c-971e-49cf-afea-0cdecd626fc6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:03:30.875520 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09797c-971e-49cf-afea-0cdecd626fc6 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:03:30.885425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1182/2362] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:03:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:03:40.881231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=63 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:03:54.909052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94986274-c8bc-48a2-8dba-58ee032c3d22 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:03:54.910488 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-94986274-c8bc-48a2-8dba-58ee032c3d22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:03:54.995640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94986274-c8bc-48a2-8dba-58ee032c3d22 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:03:54.999327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8f199-bd5d-4ed8-94ea-6ddb875abfa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:03:54.999966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8f199-bd5d-4ed8-94ea-6ddb875abfa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:03:55.000246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8f199-bd5d-4ed8-94ea-6ddb875abfa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:03:55.000549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8f199-bd5d-4ed8-94ea-6ddb875abfa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:03:55.002284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-dfa8f199-bd5d-4ed8-94ea-6ddb875abfa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:03:55.022325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8f199-bd5d-4ed8-94ea-6ddb875abfa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:03:55.023734 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8f199-bd5d-4ed8-94ea-6ddb875abfa0 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:03:55.044486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1181/2363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:03:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:03:56.225733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09797c-971e-49cf-afea-0cdecd626fc6 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:03:56.243839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09797c-971e-49cf-afea-0cdecd626fc6 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:03:56.252744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf8f38-e9ed-4d87-b39b-6118e8174034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:03:56.253934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf8f38-e9ed-4d87-b39b-6118e8174034 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:03:56.254307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf8f38-e9ed-4d87-b39b-6118e8174034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:03:56.254647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf8f38-e9ed-4d87-b39b-6118e8174034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:03:56.257256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f7cf8f38-e9ed-4d87-b39b-6118e8174034 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:03:56.277182 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf8f38-e9ed-4d87-b39b-6118e8174034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:03:56.279627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf8f38-e9ed-4d87-b39b-6118e8174034 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:03:56.295242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1183/2364] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:03:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:03:59.261516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad90688-9701-481a-90a6-508c59a4d7f9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:03:59.262036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad90688-9701-481a-90a6-508c59a4d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:03:59.262152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad90688-9701-481a-90a6-508c59a4d7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:03:59.278627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7ad90688-9701-481a-90a6-508c59a4d7f9 None None] MFA Rules not processed for user `d08a5eaa6d814e388b0af0581a530888`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:03:59.280373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7ad90688-9701-481a-90a6-508c59a4d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:03:59.335975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1182/2365] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:03:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:03:59.631567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf8f38-e9ed-4d87-b39b-6118e8174034 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:03:59.640110 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf8f38-e9ed-4d87-b39b-6118e8174034 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:03:59.643998 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e07d7-6f7e-47bb-9369-6722778dc230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:03:59.644672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e07d7-6f7e-47bb-9369-6722778dc230 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:03:59.644955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e07d7-6f7e-47bb-9369-6722778dc230 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:03:59.645231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e07d7-6f7e-47bb-9369-6722778dc230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:03:59.646967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-186e07d7-6f7e-47bb-9369-6722778dc230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:03:59.656355 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e07d7-6f7e-47bb-9369-6722778dc230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:03:59.657760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e07d7-6f7e-47bb-9369-6722778dc230 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:03:59.665487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1184/2366] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:03:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:04:09.329336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=71 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:04:09.662028 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=12 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:05:02.121725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad90688-9701-481a-90a6-508c59a4d7f9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:05:02.123121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7ad90688-9701-481a-90a6-508c59a4d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:05:02.200346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad90688-9701-481a-90a6-508c59a4d7f9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:05:02.203802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cb2f1-8155-40a1-a294-cafc0d4c737c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:05:02.204473 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cb2f1-8155-40a1-a294-cafc0d4c737c service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:05:02.204756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cb2f1-8155-40a1-a294-cafc0d4c737c service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:05:02.205033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cb2f1-8155-40a1-a294-cafc0d4c737c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:05:02.206774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a62cb2f1-8155-40a1-a294-cafc0d4c737c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:05:02.265149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cb2f1-8155-40a1-a294-cafc0d4c737c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:05:02.266611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cb2f1-8155-40a1-a294-cafc0d4c737c service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:05:02.275447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1183/2367] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:05:02.317966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e07d7-6f7e-47bb-9369-6722778dc230 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:05:02.327953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e07d7-6f7e-47bb-9369-6722778dc230 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:05:02.331826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53193eed-34fe-4e13-8410-0d997e4b5803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:05:02.332623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53193eed-34fe-4e13-8410-0d997e4b5803 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:05:02.332906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53193eed-34fe-4e13-8410-0d997e4b5803 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:05:02.333186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53193eed-34fe-4e13-8410-0d997e4b5803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:05:02.334972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-53193eed-34fe-4e13-8410-0d997e4b5803 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:05:02.394399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53193eed-34fe-4e13-8410-0d997e4b5803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:05:02.395822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53193eed-34fe-4e13-8410-0d997e4b5803 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:05:02.404686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1185/2368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:05:02.447596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cb2f1-8155-40a1-a294-cafc0d4c737c service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:05:02.457334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cb2f1-8155-40a1-a294-cafc0d4c737c service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:05:02.460959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374ce08-23ce-43be-9deb-3d56181ab10e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:05:02.461590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374ce08-23ce-43be-9deb-3d56181ab10e service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:05:02.461870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374ce08-23ce-43be-9deb-3d56181ab10e service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:05:02.462142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374ce08-23ce-43be-9deb-3d56181ab10e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:05:02.463905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2374ce08-23ce-43be-9deb-3d56181ab10e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:05:02.522081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374ce08-23ce-43be-9deb-3d56181ab10e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:05:02.523491 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374ce08-23ce-43be-9deb-3d56181ab10e service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:05:02.532280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1184/2369] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:05:12.400528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=28 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:05:12.528280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=85 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:05:29.445317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53193eed-34fe-4e13-8410-0d997e4b5803 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:05:29.446734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-53193eed-34fe-4e13-8410-0d997e4b5803 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:05:29.479653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53193eed-34fe-4e13-8410-0d997e4b5803 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:05:29.483564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969e9f9-d465-460f-86e2-0c06fe8cf1b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:05:29.484196 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969e9f9-d465-460f-86e2-0c06fe8cf1b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:05:29.484508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969e9f9-d465-460f-86e2-0c06fe8cf1b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:05:29.484788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969e9f9-d465-460f-86e2-0c06fe8cf1b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:05:29.487187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e969e9f9-d465-460f-86e2-0c06fe8cf1b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:05:29.496495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969e9f9-d465-460f-86e2-0c06fe8cf1b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:05:29.497930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969e9f9-d465-460f-86e2-0c06fe8cf1b9 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:05:29.506157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1186/2370] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:05:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:05:39.502697 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=15 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:06:01.042704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374ce08-23ce-43be-9deb-3d56181ab10e service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:06:01.044138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2374ce08-23ce-43be-9deb-3d56181ab10e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:06:01.056129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374ce08-23ce-43be-9deb-3d56181ab10e service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:06:01.059713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68d626-5eae-4609-9bcb-c3fbac35239f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:06:01.060383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68d626-5eae-4609-9bcb-c3fbac35239f service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:06:01.060668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68d626-5eae-4609-9bcb-c3fbac35239f service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:06:01.060947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68d626-5eae-4609-9bcb-c3fbac35239f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:06:01.062651 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ff68d626-5eae-4609-9bcb-c3fbac35239f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:06:01.095889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68d626-5eae-4609-9bcb-c3fbac35239f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:06:01.097518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68d626-5eae-4609-9bcb-c3fbac35239f service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:06:01.115722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1185/2371] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:06:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:06:11.113360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=19 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:07:04.138738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3868c2-70d3-4ed7-a21a-5d4ee45f5dd4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:04.139272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3868c2-70d3-4ed7-a21a-5d4ee45f5dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:04.139390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3868c2-70d3-4ed7-a21a-5d4ee45f5dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:04.165335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-da3868c2-70d3-4ed7-a21a-5d4ee45f5dd4 None None] MFA Rules not processed for user `ac4cfa3ccdab43c3adafcef62e62365f`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:07:04.166528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-da3868c2-70d3-4ed7-a21a-5d4ee45f5dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:04.173799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1187/2372] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:07:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:07:04.411457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68d626-5eae-4609-9bcb-c3fbac35239f service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:07:04.421098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68d626-5eae-4609-9bcb-c3fbac35239f service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:07:04.425062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cf13c-81cd-4b8a-99ba-0b4e0720db4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:07:04.425703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cf13c-81cd-4b8a-99ba-0b4e0720db4b service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:04.426004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cf13c-81cd-4b8a-99ba-0b4e0720db4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:04.426345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cf13c-81cd-4b8a-99ba-0b4e0720db4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:04.428094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-860cf13c-81cd-4b8a-99ba-0b4e0720db4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:04.437843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cf13c-81cd-4b8a-99ba-0b4e0720db4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:07:04.439256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cf13c-81cd-4b8a-99ba-0b4e0720db4b service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:07:04.447226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1186/2373] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:07:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:07:04.760812 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3868c2-70d3-4ed7-a21a-5d4ee45f5dd4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:07:04.774093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3868c2-70d3-4ed7-a21a-5d4ee45f5dd4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:07:04.779134 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbc691-b5ef-4624-9e09-28357e931a54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:07:04.779827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbc691-b5ef-4624-9e09-28357e931a54 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:04.780115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbc691-b5ef-4624-9e09-28357e931a54 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:04.780486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbc691-b5ef-4624-9e09-28357e931a54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:04.782266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-49bbc691-b5ef-4624-9e09-28357e931a54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:04.794502 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbc691-b5ef-4624-9e09-28357e931a54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:07:04.796200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbc691-b5ef-4624-9e09-28357e931a54 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:07:04.807347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1188/2374] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:07:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:07:05.783112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cf13c-81cd-4b8a-99ba-0b4e0720db4b service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:07:05.786194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-860cf13c-81cd-4b8a-99ba-0b4e0720db4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:05.922477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cf13c-81cd-4b8a-99ba-0b4e0720db4b service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:07:05.926571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141e724-b24c-4506-a763-2bf320782787 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e07c59babad479cadbf74f15ee5eea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:07:05.927468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141e724-b24c-4506-a763-2bf320782787 service glance] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:05.927625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141e724-b24c-4506-a763-2bf320782787 service glance] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:05.928023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141e724-b24c-4506-a763-2bf320782787 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:05.938674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a141e724-b24c-4506-a763-2bf320782787 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a141e724-b24c-4506-a763-2bf320782787 service glance] Could not find project: 66ee4df184ae4eb6a9a559fd40a7ecc1.: keystone.exception.ProjectNotFound: Could not find project: 66ee4df184ae4eb6a9a559fd40a7ecc1. Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 21 18:07:05.947462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 18:07:05.948751 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 18:07:05.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 18:07:05.951236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 18:07:05.952471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 66ee4df184ae4eb6a9a559fd40a7ecc1. Jul 21 18:07:05.952471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 21 18:07:05.952471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1187/2375] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:07:05 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Jul 21 18:07:07.810010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbc691-b5ef-4624-9e09-28357e931a54 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:07:07.811504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-49bbc691-b5ef-4624-9e09-28357e931a54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:07.844906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbc691-b5ef-4624-9e09-28357e931a54 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:07:07.850921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763085-6fa0-405a-b966-26af9a9ef94f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:07:07.851571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763085-6fa0-405a-b966-26af9a9ef94f service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:07.851872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763085-6fa0-405a-b966-26af9a9ef94f service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:07.852151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763085-6fa0-405a-b966-26af9a9ef94f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:07.861468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763085-6fa0-405a-b966-26af9a9ef94f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:07:07.863071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763085-6fa0-405a-b966-26af9a9ef94f service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:07:07.873880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1189/2376] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:07:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:07:13.966528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141e724-b24c-4506-a763-2bf320782787 service glance] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:07:13.986726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141e724-b24c-4506-a763-2bf320782787 service glance] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:07:13.994090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68df7d7-b6be-4355-a690-92e3576c377e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:07:13.994848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68df7d7-b6be-4355-a690-92e3576c377e service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:13.995242 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68df7d7-b6be-4355-a690-92e3576c377e service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:13.995591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68df7d7-b6be-4355-a690-92e3576c377e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:14.017508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68df7d7-b6be-4355-a690-92e3576c377e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:07:14.017508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68df7d7-b6be-4355-a690-92e3576c377e service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:07:14.030399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1188/2377] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:07:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:07:17.869263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=23 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:07:20.105997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204984a0-610f-4486-92a9-ad59919fbf17 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:20.106514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204984a0-610f-4486-92a9-ad59919fbf17 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:20.106614 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204984a0-610f-4486-92a9-ad59919fbf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:20.170202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-204984a0-610f-4486-92a9-ad59919fbf17 None None] MFA Rules not processed for user `75f11eaa4b504d0aa4018fb4ffbbd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:07:20.171443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-204984a0-610f-4486-92a9-ad59919fbf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:20.242516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1190/2378] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 18:07:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:07:20.289484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68df7d7-b6be-4355-a690-92e3576c377e service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:07:20.290928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a68df7d7-b6be-4355-a690-92e3576c377e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:20.387367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68df7d7-b6be-4355-a690-92e3576c377e service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:07:20.392620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfa073-44bf-4d2d-98f2-dbdfdc3b4dc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a26412cf9f64b1bbbd8e595b55bcebc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:07:20.393275 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfa073-44bf-4d2d-98f2-dbdfdc3b4dc3 service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:20.394082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfa073-44bf-4d2d-98f2-dbdfdc3b4dc3 service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:20.394366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfa073-44bf-4d2d-98f2-dbdfdc3b4dc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:20.403914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfa073-44bf-4d2d-98f2-dbdfdc3b4dc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:07:20.405614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfa073-44bf-4d2d-98f2-dbdfdc3b4dc3 service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:07:20.414792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1189/2379] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jul 21 18:07:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 18:07:30.239470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=44 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:07:30.410440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=86 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:07:39.243845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b71a5-2d21-4fc7-9574-dd755d4c5c4f None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:39.244374 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b71a5-2d21-4fc7-9574-dd755d4c5c4f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:39.244533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b71a5-2d21-4fc7-9574-dd755d4c5c4f None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:39.247229 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1191/2380] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 18:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 18:07:39.255821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd88ce-16cf-448c-a319-3f516fb64a49 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:39.256373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd88ce-16cf-448c-a319-3f516fb64a49 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:39.256505 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd88ce-16cf-448c-a319-3f516fb64a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:39.313944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b4cd88ce-16cf-448c-a319-3f516fb64a49 None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:07:39.316083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b4cd88ce-16cf-448c-a319-3f516fb64a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:39.325672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1190/2381] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 18:07:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:07:39.337902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b71a5-2d21-4fc7-9574-dd755d4c5c4f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:07:39.339582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9f0b71a5-2d21-4fc7-9574-dd755d4c5c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:39.352603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b71a5-2d21-4fc7-9574-dd755d4c5c4f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:07:39.356238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842eed27-55df-4e3d-9450-88621bdb6b58 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e07c59babad479cadbf74f15ee5eea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:07:39.356906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842eed27-55df-4e3d-9450-88621bdb6b58 service glance] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:39.357185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842eed27-55df-4e3d-9450-88621bdb6b58 service glance] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:39.357462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842eed27-55df-4e3d-9450-88621bdb6b58 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:39.368647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842eed27-55df-4e3d-9450-88621bdb6b58 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:07:39.370145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842eed27-55df-4e3d-9450-88621bdb6b58 service glance] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:07:39.381546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1192/2382] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:07:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:07:39.453238 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c755b-d2e4-40f7-9f48-066f33bc487f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:39.453876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c755b-d2e4-40f7-9f48-066f33bc487f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:39.454160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c755b-d2e4-40f7-9f48-066f33bc487f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:39.456575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1191/2383] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 18:07:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 18:07:39.461712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e3b91-1ca4-49fb-a45d-262db7f8ee0b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:39.462005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e3b91-1ca4-49fb-a45d-262db7f8ee0b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:39.462359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e3b91-1ca4-49fb-a45d-262db7f8ee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:39.502926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a28e3b91-1ca4-49fb-a45d-262db7f8ee0b None None] MFA Rules not processed for user `b0edda384caf46819169b69b858afc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:07:39.504200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a28e3b91-1ca4-49fb-a45d-262db7f8ee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:39.515024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1193/2384] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 18:07:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:07:39.527262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c755b-d2e4-40f7-9f48-066f33bc487f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:07:39.541355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c755b-d2e4-40f7-9f48-066f33bc487f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:07:39.545226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9bdaa-2936-4b3d-8389-429590386ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:07:39.545893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9bdaa-2936-4b3d-8389-429590386ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:39.546177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9bdaa-2936-4b3d-8389-429590386ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:39.546584 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9bdaa-2936-4b3d-8389-429590386ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:39.556259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9bdaa-2936-4b3d-8389-429590386ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:07:39.557907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9bdaa-2936-4b3d-8389-429590386ba4 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:07:39.567462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1192/2385] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:07:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:07:39.656864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cca676-1974-4496-998b-b9d983724500 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:39.657159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cca676-1974-4496-998b-b9d983724500 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:39.657437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cca676-1974-4496-998b-b9d983724500 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:39.658997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1194/2386] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 18:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 18:07:39.665295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03500ba9-9f78-4601-92a8-22e0d02a6445 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:39.665588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03500ba9-9f78-4601-92a8-22e0d02a6445 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:39.665871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03500ba9-9f78-4601-92a8-22e0d02a6445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:39.685648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-03500ba9-9f78-4601-92a8-22e0d02a6445 None None] MFA Rules not processed for user `380d6fe7057d44bfb20b0ae2bb1e664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:07:39.686874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-03500ba9-9f78-4601-92a8-22e0d02a6445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:39.696906 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1193/2387] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 18:07:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:07:39.707633 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cca676-1974-4496-998b-b9d983724500 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:07:39.719321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cca676-1974-4496-998b-b9d983724500 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:07:39.723419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527080c6-72d9-49d2-adfe-67ab19e4bf2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:07:39.724065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527080c6-72d9-49d2-adfe-67ab19e4bf2a service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:39.724388 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527080c6-72d9-49d2-adfe-67ab19e4bf2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:39.724670 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527080c6-72d9-49d2-adfe-67ab19e4bf2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:39.735227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527080c6-72d9-49d2-adfe-67ab19e4bf2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:07:39.737224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527080c6-72d9-49d2-adfe-67ab19e4bf2a service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:07:39.748011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1195/2388] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:07:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:07:42.898835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2e29c-3ff6-42ed-9582-5a282cb6264a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:42.899680 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2e29c-3ff6-42ed-9582-5a282cb6264a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:42.900181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2e29c-3ff6-42ed-9582-5a282cb6264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:42.927460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3ae2e29c-3ff6-42ed-9582-5a282cb6264a None None] MFA Rules not processed for user `75f11eaa4b504d0aa4018fb4ffbbd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:07:42.938903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3ae2e29c-3ff6-42ed-9582-5a282cb6264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:07:42.956698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1194/2389] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 18:07:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:07:42.965209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527080c6-72d9-49d2-adfe-67ab19e4bf2a service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:07:42.981634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527080c6-72d9-49d2-adfe-67ab19e4bf2a service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:07:42.996672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a8434-0815-47d2-b631-32008fd369cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a26412cf9f64b1bbbd8e595b55bcebc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:07:42.998055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a8434-0815-47d2-b631-32008fd369cd service swift] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:07:42.998356 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a8434-0815-47d2-b631-32008fd369cd service swift] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:07:42.998662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a8434-0815-47d2-b631-32008fd369cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:07:43.013194 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a8434-0815-47d2-b631-32008fd369cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:07:43.015092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a8434-0815-47d2-b631-32008fd369cd service swift] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:07:43.029816 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1196/2390] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jul 21 18:07:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 18:07:52.920855 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=25 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:07:53.022007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=27 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:02.809771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c8442-833d-4c2b-b216-a18aa0493a42 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:02.810264 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c8442-833d-4c2b-b216-a18aa0493a42 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:02.810416 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c8442-833d-4c2b-b216-a18aa0493a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:02.829396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-487c8442-833d-4c2b-b216-a18aa0493a42 None None] MFA Rules not processed for user `75f11eaa4b504d0aa4018fb4ffbbd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:02.830595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-487c8442-833d-4c2b-b216-a18aa0493a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:02.837890 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1195/2391] 10.222.0.57 () {58 vars in 967 bytes} [Thu Jul 21 18:08:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:02.851902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a8434-0815-47d2-b631-32008fd369cd service swift] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:02.862309 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a8434-0815-47d2-b631-32008fd369cd service swift] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:02.868089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22800fc3-da0d-42b0-8545-1d1a65808d0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a26412cf9f64b1bbbd8e595b55bcebc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:02.868780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22800fc3-da0d-42b0-8545-1d1a65808d0f service swift] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:02.869080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22800fc3-da0d-42b0-8545-1d1a65808d0f service swift] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:02.869339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22800fc3-da0d-42b0-8545-1d1a65808d0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:02.877564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22800fc3-da0d-42b0-8545-1d1a65808d0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:02.879140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22800fc3-da0d-42b0-8545-1d1a65808d0f service swift] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:02.889199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1197/2392] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jul 21 18:08:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 18:08:12.823807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:12.885402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:25.854220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c8442-833d-4c2b-b216-a18aa0493a42 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:25.863731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c8442-833d-4c2b-b216-a18aa0493a42 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:25.868118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39157042-34b1-470d-a2da-1ee48cd7bd7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:25.868861 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39157042-34b1-470d-a2da-1ee48cd7bd7c service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:25.869166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39157042-34b1-470d-a2da-1ee48cd7bd7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:25.869443 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39157042-34b1-470d-a2da-1ee48cd7bd7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:25.871178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-39157042-34b1-470d-a2da-1ee48cd7bd7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:25.958386 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39157042-34b1-470d-a2da-1ee48cd7bd7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:25.960547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39157042-34b1-470d-a2da-1ee48cd7bd7c service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:25.979735 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1196/2393] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:08:26.561524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e3cfc-f4f6-4567-b835-a352cfab6575 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:26.562064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e3cfc-f4f6-4567-b835-a352cfab6575 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:26.562064 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e3cfc-f4f6-4567-b835-a352cfab6575 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:26.563489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1198/2394] 10.222.0.57 () {54 vars in 921 bytes} [Thu Jul 21 18:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 18:08:26.572366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa406081-f294-4de6-a507-5190d30c3961 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:26.572738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa406081-f294-4de6-a507-5190d30c3961 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:26.573088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa406081-f294-4de6-a507-5190d30c3961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:26.610869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-aa406081-f294-4de6-a507-5190d30c3961 None None] MFA Rules not processed for user `901cf70064c34b63a0eda979a5ce77e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:26.612132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aa406081-f294-4de6-a507-5190d30c3961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:26.622218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1197/2395] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Jul 21 18:08:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:26.628282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e3cfc-f4f6-4567-b835-a352cfab6575 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:26.630251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dc1e3cfc-f4f6-4567-b835-a352cfab6575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:26.638590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa406081-f294-4de6-a507-5190d30c3961 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:26.640361 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aa406081-f294-4de6-a507-5190d30c3961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:26.653511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e3cfc-f4f6-4567-b835-a352cfab6575 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:26.653785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa406081-f294-4de6-a507-5190d30c3961 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:26.658119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:26.658693 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a6dcb-a8c7-47ae-b956-516dcf847aed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:26.659166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:26.659336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a6dcb-a8c7-47ae-b956-516dcf847aed service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:26.659566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:26.659679 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a6dcb-a8c7-47ae-b956-516dcf847aed service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:26.659877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a6dcb-a8c7-47ae-b956-516dcf847aed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:26.660022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] PATH_INFO: `/v3/users/ac4cfa3ccdab43c3adafcef62e62365f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:26.661288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac4cfa3ccdab43c3adafcef62e62365f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:26.663507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:26.671117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a6dcb-a8c7-47ae-b956-516dcf847aed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:26.672846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a6dcb-a8c7-47ae-b956-516dcf847aed service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:26.683818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1198/2396] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:08:26.712865 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac4cfa3ccdab43c3adafcef62e62365f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:26.720185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac4cfa3ccdab43c3adafcef62e62365f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:26.720888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac4cfa3ccdab43c3adafcef62e62365f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:26.732838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac4cfa3ccdab43c3adafcef62e62365f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:26.741569 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-69ade4a8-c25e-4346-bcb4-094795097063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac4cfa3ccdab43c3adafcef62e62365f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:26.747658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1199/2397] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:08:26 2022] DELETE /identity/v3/users/ac4cfa3ccdab43c3adafcef62e62365f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:26.916143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a6dcb-a8c7-47ae-b956-516dcf847aed service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:26.927125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a6dcb-a8c7-47ae-b956-516dcf847aed service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:26.992070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:26.992969 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:26.993250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:26.993544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] PATH_INFO: `/v3/projects/e47e6891d6044e2caf067cad4309c57f` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:26.994709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e47e6891d6044e2caf067cad4309c57f)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:26.996645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:27.032406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:27.043103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e47e6891d6044e2caf067cad4309c57f', 'request_id': 'req-4633f432-eca8-44aa-ada0-cce4487efe86'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:27.050335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e47e6891d6044e2caf067cad4309c57f', 'request_id': 'req-4633f432-eca8-44aa-ada0-cce4487efe86'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:27.056951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e47e6891d6044e2caf067cad4309c57f', 'request_id': 'req-4633f432-eca8-44aa-ada0-cce4487efe86'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:27.067782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] The token cache is being invalidate because project e47e6891d6044e2caf067cad4309c57f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:08:27.068119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:27.068790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4633f432-eca8-44aa-ada0-cce4487efe86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:27.070377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1199/2398] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:08:26 2022] DELETE /identity/v3/projects/e47e6891d6044e2caf067cad4309c57f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:27.075304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45aae0-9230-43b2-b145-f1446e0f28e8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:27.075596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45aae0-9230-43b2-b145-f1446e0f28e8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:27.075878 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45aae0-9230-43b2-b145-f1446e0f28e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:27.121208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-cd45aae0-9230-43b2-b145-f1446e0f28e8 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:27.122405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cd45aae0-9230-43b2-b145-f1446e0f28e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:27.181856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1200/2399] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:27.188954 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead6269-5b6f-4431-aa11-a0cd6dc27314 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:27.189247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead6269-5b6f-4431-aa11-a0cd6dc27314 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:27.189528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead6269-5b6f-4431-aa11-a0cd6dc27314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:27.211450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-dead6269-5b6f-4431-aa11-a0cd6dc27314 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:27.212806 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-dead6269-5b6f-4431-aa11-a0cd6dc27314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:27.228002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1200/2400] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:27.232282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45aae0-9230-43b2-b145-f1446e0f28e8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:27.243348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45aae0-9230-43b2-b145-f1446e0f28e8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:27.248067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd9abc-ac30-4144-9006-14aee7e31eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:27.248756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd9abc-ac30-4144-9006-14aee7e31eaf admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:27.249043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd9abc-ac30-4144-9006-14aee7e31eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:27.249322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd9abc-ac30-4144-9006-14aee7e31eaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:27.250048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd9abc-ac30-4144-9006-14aee7e31eaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:08:27.250358 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd9abc-ac30-4144-9006-14aee7e31eaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:27.251982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd9abc-ac30-4144-9006-14aee7e31eaf admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:27.264775 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1201/2401] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 18:08:27.268864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead6269-5b6f-4431-aa11-a0cd6dc27314 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:27.279108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead6269-5b6f-4431-aa11-a0cd6dc27314 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:27.283768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e605d51-5445-4176-bf62-17a97cfe5f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:27.284451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e605d51-5445-4176-bf62-17a97cfe5f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:27.284736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e605d51-5445-4176-bf62-17a97cfe5f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:27.285012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e605d51-5445-4176-bf62-17a97cfe5f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:27.285713 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e605d51-5445-4176-bf62-17a97cfe5f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:27.288212 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e605d51-5445-4176-bf62-17a97cfe5f4c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:27.323813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1201/2402] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:08:27 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:08:27.327973 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd9abc-ac30-4144-9006-14aee7e31eaf admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:27.342325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd9abc-ac30-4144-9006-14aee7e31eaf admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:27.375631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:27.401239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285873e-a84a-4788-8db6-2a171584116e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:27.401901 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285873e-a84a-4788-8db6-2a171584116e admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:27.402184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285873e-a84a-4788-8db6-2a171584116e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:27.402459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285873e-a84a-4788-8db6-2a171584116e admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:27.403147 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285873e-a84a-4788-8db6-2a171584116e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:27.405105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285873e-a84a-4788-8db6-2a171584116e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:27.425049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1202/2403] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:08:27 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:08:27.429923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e605d51-5445-4176-bf62-17a97cfe5f4c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:27.439994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e605d51-5445-4176-bf62-17a97cfe5f4c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:27.445258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc24f7e-f489-4730-b50c-04ec632669f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:27.445912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc24f7e-f489-4730-b50c-04ec632669f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:27.446190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc24f7e-f489-4730-b50c-04ec632669f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:27.446463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc24f7e-f489-4730-b50c-04ec632669f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:27.447174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc24f7e-f489-4730-b50c-04ec632669f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:27.448799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc24f7e-f489-4730-b50c-04ec632669f8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:27.458506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1202/2404] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:08:27.462329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285873e-a84a-4788-8db6-2a171584116e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:27.472145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285873e-a84a-4788-8db6-2a171584116e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:27.477026 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59e5d5-0ceb-4fcd-b920-4b380bd56674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:27.477672 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59e5d5-0ceb-4fcd-b920-4b380bd56674 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:27.477953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59e5d5-0ceb-4fcd-b920-4b380bd56674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:27.478227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59e5d5-0ceb-4fcd-b920-4b380bd56674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:27.478948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59e5d5-0ceb-4fcd-b920-4b380bd56674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:27.480557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59e5d5-0ceb-4fcd-b920-4b380bd56674 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:27.490940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1203/2405] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:08:27.500141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc24f7e-f489-4730-b50c-04ec632669f8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:27.510815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc24f7e-f489-4730-b50c-04ec632669f8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:27.515388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e824c53-afe9-46d6-a79e-6d206f01dcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:27.516256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e824c53-afe9-46d6-a79e-6d206f01dcc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:27.516562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e824c53-afe9-46d6-a79e-6d206f01dcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:27.516834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e824c53-afe9-46d6-a79e-6d206f01dcc9 admin admin] PATH_INFO: `/v3/projects/076a5b3e4cd141b9bd9ef199e0fa0ba1/users/293d4ca894e24e4fb6c4e7aa14256d78/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:27.531293 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e824c53-afe9-46d6-a79e-6d206f01dcc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076a5b3e4cd141b9bd9ef199e0fa0ba1, user_id=293d4ca894e24e4fb6c4e7aa14256d78, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:27.534069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e824c53-afe9-46d6-a79e-6d206f01dcc9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:27.546811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1203/2406] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:08:27 2022] PUT /identity/v3/projects/076a5b3e4cd141b9bd9ef199e0fa0ba1/users/293d4ca894e24e4fb6c4e7aa14256d78/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:27.551933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f85c7-9801-4e64-bbd8-896fe5095221 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:27.553007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f85c7-9801-4e64-bbd8-896fe5095221 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:27.553290 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f85c7-9801-4e64-bbd8-896fe5095221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:27.571363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2e4f85c7-9801-4e64-bbd8-896fe5095221 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:27.572591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2e4f85c7-9801-4e64-bbd8-896fe5095221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:27.630332 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1204/2407] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:27.640733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e824c53-afe9-46d6-a79e-6d206f01dcc9 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:27.642447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8e824c53-afe9-46d6-a79e-6d206f01dcc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:27.698151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e824c53-afe9-46d6-a79e-6d206f01dcc9 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:27.701719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91740804-5a5a-4fa0-aa95-b46e1133bd35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:27.702351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91740804-5a5a-4fa0-aa95-b46e1133bd35 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:27.702630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91740804-5a5a-4fa0-aa95-b46e1133bd35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:27.702904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91740804-5a5a-4fa0-aa95-b46e1133bd35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:27.711796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91740804-5a5a-4fa0-aa95-b46e1133bd35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:27.713241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91740804-5a5a-4fa0-aa95-b46e1133bd35 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:27.722378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1204/2408] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:08:31.778892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfaf28-83ee-49ec-bb51-e9549c318f46 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:31.779743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfaf28-83ee-49ec-bb51-e9549c318f46 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:31.780208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfaf28-83ee-49ec-bb51-e9549c318f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:31.812096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f2dfaf28-83ee-49ec-bb51-e9549c318f46 None None] MFA Rules not processed for user `293d4ca894e24e4fb6c4e7aa14256d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:31.813612 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f2dfaf28-83ee-49ec-bb51-e9549c318f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:31.887861 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1205/2409] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:31.892688 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91740804-5a5a-4fa0-aa95-b46e1133bd35 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:31.903585 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91740804-5a5a-4fa0-aa95-b46e1133bd35 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:31.908268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87fe95-fcc6-4644-87ad-ce2886e5629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:31.908960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87fe95-fcc6-4644-87ad-ce2886e5629c admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:31.909241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87fe95-fcc6-4644-87ad-ce2886e5629c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:31.909516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87fe95-fcc6-4644-87ad-ce2886e5629c admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:31.910219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87fe95-fcc6-4644-87ad-ce2886e5629c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:31.912318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87fe95-fcc6-4644-87ad-ce2886e5629c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:31.948513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1205/2410] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:08:31 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:08:31.952169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfaf28-83ee-49ec-bb51-e9549c318f46 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:31.963433 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfaf28-83ee-49ec-bb51-e9549c318f46 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:32.021260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:32.035824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e5c54-850b-45d4-920c-b069b125e68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:32.036618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e5c54-850b-45d4-920c-b069b125e68a admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:32.036946 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e5c54-850b-45d4-920c-b069b125e68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:32.037192 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e5c54-850b-45d4-920c-b069b125e68a admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:32.037908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e5c54-850b-45d4-920c-b069b125e68a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:32.039888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e5c54-850b-45d4-920c-b069b125e68a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:32.062553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1206/2411] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:08:31 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:08:32.066166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87fe95-fcc6-4644-87ad-ce2886e5629c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:32.077471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87fe95-fcc6-4644-87ad-ce2886e5629c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:32.082040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b008a77-84e6-4019-aec6-33dd566949a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:32.082703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b008a77-84e6-4019-aec6-33dd566949a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:32.082985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b008a77-84e6-4019-aec6-33dd566949a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:32.083277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b008a77-84e6-4019-aec6-33dd566949a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:32.084534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b008a77-84e6-4019-aec6-33dd566949a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:32.086345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b008a77-84e6-4019-aec6-33dd566949a3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:32.097462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1206/2412] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:08:32.101776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e5c54-850b-45d4-920c-b069b125e68a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:32.111822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e5c54-850b-45d4-920c-b069b125e68a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:32.117260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5217822-2cc6-44f4-bcf0-040805510b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:32.118142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5217822-2cc6-44f4-bcf0-040805510b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:32.118424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5217822-2cc6-44f4-bcf0-040805510b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:32.118698 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5217822-2cc6-44f4-bcf0-040805510b33 admin admin] PATH_INFO: `/v3/projects/c77d072230894424a78dc0a43457c8da/users/fd67132c544c46c9a7d82f55128bdcb3/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:32.134373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5217822-2cc6-44f4-bcf0-040805510b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77d072230894424a78dc0a43457c8da, user_id=fd67132c544c46c9a7d82f55128bdcb3, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:32.137282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5217822-2cc6-44f4-bcf0-040805510b33 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:32.150599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1207/2413] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:08:32 2022] PUT /identity/v3/projects/c77d072230894424a78dc0a43457c8da/users/fd67132c544c46c9a7d82f55128bdcb3/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:32.154557 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b008a77-84e6-4019-aec6-33dd566949a3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:32.165351 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b008a77-84e6-4019-aec6-33dd566949a3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:32.183934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:32.226860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8016eb-084f-4723-9bc3-6657dc0d5acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:32.227527 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8016eb-084f-4723-9bc3-6657dc0d5acc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:32.227807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8016eb-084f-4723-9bc3-6657dc0d5acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:32.228080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8016eb-084f-4723-9bc3-6657dc0d5acc admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:32.228832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8016eb-084f-4723-9bc3-6657dc0d5acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:32.230404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8016eb-084f-4723-9bc3-6657dc0d5acc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:32.241432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1207/2414] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:08:32.245666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5217822-2cc6-44f4-bcf0-040805510b33 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:32.256655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5217822-2cc6-44f4-bcf0-040805510b33 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:32.261164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa432ccd-0172-4be0-a917-918f300e8783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:32.262061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa432ccd-0172-4be0-a917-918f300e8783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:32.262343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa432ccd-0172-4be0-a917-918f300e8783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:32.262618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa432ccd-0172-4be0-a917-918f300e8783 admin admin] PATH_INFO: `/v3/projects/c77d072230894424a78dc0a43457c8da/users/fd67132c544c46c9a7d82f55128bdcb3/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:32.264599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa432ccd-0172-4be0-a917-918f300e8783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77d072230894424a78dc0a43457c8da, user_id=fd67132c544c46c9a7d82f55128bdcb3, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:32.267251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa432ccd-0172-4be0-a917-918f300e8783 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:32.277743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1208/2415] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:08:32 2022] PUT /identity/v3/projects/c77d072230894424a78dc0a43457c8da/users/fd67132c544c46c9a7d82f55128bdcb3/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:33.722408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8016eb-084f-4723-9bc3-6657dc0d5acc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:33.724272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2d8016eb-084f-4723-9bc3-6657dc0d5acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:33.810184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8016eb-084f-4723-9bc3-6657dc0d5acc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:33.886863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6b556-e3f8-4334-b755-7a235782e699 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:33.887570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6b556-e3f8-4334-b755-7a235782e699 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:33.887833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6b556-e3f8-4334-b755-7a235782e699 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:33.888175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6b556-e3f8-4334-b755-7a235782e699 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:33.890106 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-93c6b556-e3f8-4334-b755-7a235782e699 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:33.902877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6b556-e3f8-4334-b755-7a235782e699 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:33.904586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6b556-e3f8-4334-b755-7a235782e699 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:33.923965 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1208/2416] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:08:37.697954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e862e2a-21a8-4c82-b397-89813714bbe7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:37.698767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e862e2a-21a8-4c82-b397-89813714bbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:37.700009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e862e2a-21a8-4c82-b397-89813714bbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:37.756702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0e862e2a-21a8-4c82-b397-89813714bbe7 None None] MFA Rules not processed for user `fd67132c544c46c9a7d82f55128bdcb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:37.758006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0e862e2a-21a8-4c82-b397-89813714bbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:37.886891 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1209/2417] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:37.897126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6b556-e3f8-4334-b755-7a235782e699 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:37.901993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-93c6b556-e3f8-4334-b755-7a235782e699 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:38.005842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:38.046317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6b556-e3f8-4334-b755-7a235782e699 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:38.050764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193300a-9c26-4532-96a9-fd956bc0397f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:38.051429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193300a-9c26-4532-96a9-fd956bc0397f service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:38.051740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193300a-9c26-4532-96a9-fd956bc0397f service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:38.052370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193300a-9c26-4532-96a9-fd956bc0397f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:38.125650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193300a-9c26-4532-96a9-fd956bc0397f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:38.127562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193300a-9c26-4532-96a9-fd956bc0397f service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:38.140821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1209/2418] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:08:42.190911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e862e2a-21a8-4c82-b397-89813714bbe7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:42.192894 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0e862e2a-21a8-4c82-b397-89813714bbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:42.262563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e862e2a-21a8-4c82-b397-89813714bbe7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:42.266102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a5c64-570f-4d6b-826a-552ba7b81723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:42.266755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a5c64-570f-4d6b-826a-552ba7b81723 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:42.267039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a5c64-570f-4d6b-826a-552ba7b81723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:42.267317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a5c64-570f-4d6b-826a-552ba7b81723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:42.269133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-994a5c64-570f-4d6b-826a-552ba7b81723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:42.316994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:42.344779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a5c64-570f-4d6b-826a-552ba7b81723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:42.346233 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a5c64-570f-4d6b-826a-552ba7b81723 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:42.357598 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1210/2419] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:08:45.952763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193300a-9c26-4532-96a9-fd956bc0397f service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:45.967667 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193300a-9c26-4532-96a9-fd956bc0397f service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:45.972701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10eeff-7137-4dc0-ba2a-aaf3b292e63e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:45.973382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10eeff-7137-4dc0-ba2a-aaf3b292e63e service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:45.973661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10eeff-7137-4dc0-ba2a-aaf3b292e63e service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:45.973989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10eeff-7137-4dc0-ba2a-aaf3b292e63e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:45.975719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ac10eeff-7137-4dc0-ba2a-aaf3b292e63e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:45.987942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10eeff-7137-4dc0-ba2a-aaf3b292e63e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:45.989781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10eeff-7137-4dc0-ba2a-aaf3b292e63e service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:46.002004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1210/2420] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:08:46.008339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a5c64-570f-4d6b-826a-552ba7b81723 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:46.009836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-994a5c64-570f-4d6b-826a-552ba7b81723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:46.023135 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a5c64-570f-4d6b-826a-552ba7b81723 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:46.027804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:46.028719 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.029002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.029279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] PATH_INFO: `/v3/users/d08a5eaa6d814e388b0af0581a530888` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.030685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08a5eaa6d814e388b0af0581a530888)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:46.032708 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:46.088225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08a5eaa6d814e388b0af0581a530888'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:46.095219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08a5eaa6d814e388b0af0581a530888'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:46.095928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08a5eaa6d814e388b0af0581a530888'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:46.107824 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08a5eaa6d814e388b0af0581a530888'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:46.115576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6dacf4e8-12ab-448b-8782-a840591f501e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08a5eaa6d814e388b0af0581a530888'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:46.122347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1211/2421] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:08:46 2022] DELETE /identity/v3/users/d08a5eaa6d814e388b0af0581a530888 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:46.307581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10eeff-7137-4dc0-ba2a-aaf3b292e63e service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:46.321896 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10eeff-7137-4dc0-ba2a-aaf3b292e63e service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:46.381844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:46.382712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.382995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.383269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] PATH_INFO: `/v3/projects/444abcf11d564f7692af245f2bd98a84` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.384490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444abcf11d564f7692af245f2bd98a84)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:46.386565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:46.434075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444abcf11d564f7692af245f2bd98a84', 'request_id': 'req-68daee59-f431-4c22-8edd-feff7a3f8c35'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:46.442714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444abcf11d564f7692af245f2bd98a84', 'request_id': 'req-68daee59-f431-4c22-8edd-feff7a3f8c35'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:46.448378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444abcf11d564f7692af245f2bd98a84', 'request_id': 'req-68daee59-f431-4c22-8edd-feff7a3f8c35'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:46.459304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] The token cache is being invalidate because project 444abcf11d564f7692af245f2bd98a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:08:46.459594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:46.460237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-68daee59-f431-4c22-8edd-feff7a3f8c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:46.461838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1211/2422] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:08:46 2022] DELETE /identity/v3/projects/444abcf11d564f7692af245f2bd98a84 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:46.467203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa173d0d-e5d5-4684-aebb-5dfd3049ea1b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.467203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa173d0d-e5d5-4684-aebb-5dfd3049ea1b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.467436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa173d0d-e5d5-4684-aebb-5dfd3049ea1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.485992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-fa173d0d-e5d5-4684-aebb-5dfd3049ea1b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:46.487200 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fa173d0d-e5d5-4684-aebb-5dfd3049ea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:46.572348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1212/2423] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:46.576864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159c48e-1cec-4844-82c2-aec8d4be5547 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.577316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159c48e-1cec-4844-82c2-aec8d4be5547 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.577655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159c48e-1cec-4844-82c2-aec8d4be5547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.585976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:46.606918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8159c48e-1cec-4844-82c2-aec8d4be5547 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:46.608163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8159c48e-1cec-4844-82c2-aec8d4be5547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:46.617004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1212/2424] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:46.621278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa173d0d-e5d5-4684-aebb-5dfd3049ea1b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:46.631644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa173d0d-e5d5-4684-aebb-5dfd3049ea1b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:46.637181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3452f-71e5-44d9-a000-f4e1e89134a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:46.637919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3452f-71e5-44d9-a000-f4e1e89134a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.638205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3452f-71e5-44d9-a000-f4e1e89134a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.638485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3452f-71e5-44d9-a000-f4e1e89134a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.639368 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3452f-71e5-44d9-a000-f4e1e89134a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:08:46.639687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3452f-71e5-44d9-a000-f4e1e89134a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:46.641704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3452f-71e5-44d9-a000-f4e1e89134a1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:46.658851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1213/2425] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:08:46.661940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159c48e-1cec-4844-82c2-aec8d4be5547 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:46.671280 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159c48e-1cec-4844-82c2-aec8d4be5547 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:46.675848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c132e-4dec-442e-9285-34ef663f2305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:46.676542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c132e-4dec-442e-9285-34ef663f2305 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.677068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c132e-4dec-442e-9285-34ef663f2305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.677346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c132e-4dec-442e-9285-34ef663f2305 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.678040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c132e-4dec-442e-9285-34ef663f2305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:46.679941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c132e-4dec-442e-9285-34ef663f2305 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:46.705633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1213/2426] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:08:46 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:08:46.710602 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3452f-71e5-44d9-a000-f4e1e89134a1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:46.721085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3452f-71e5-44d9-a000-f4e1e89134a1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:46.752845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:46.782279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ae20d-93fa-47f5-8da2-993c86397a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:46.782952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ae20d-93fa-47f5-8da2-993c86397a85 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.783235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ae20d-93fa-47f5-8da2-993c86397a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.783516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ae20d-93fa-47f5-8da2-993c86397a85 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.784209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ae20d-93fa-47f5-8da2-993c86397a85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:46.786751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ae20d-93fa-47f5-8da2-993c86397a85 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:46.806860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1214/2427] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:08:46 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:08:46.811136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c132e-4dec-442e-9285-34ef663f2305 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:46.822420 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c132e-4dec-442e-9285-34ef663f2305 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:46.827237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a3c04-609a-471e-95ec-387fc0e13958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:46.827902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a3c04-609a-471e-95ec-387fc0e13958 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.828185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a3c04-609a-471e-95ec-387fc0e13958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.828494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a3c04-609a-471e-95ec-387fc0e13958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.829247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a3c04-609a-471e-95ec-387fc0e13958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:46.830843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a3c04-609a-471e-95ec-387fc0e13958 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:46.842365 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1214/2428] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:08:46.846278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ae20d-93fa-47f5-8da2-993c86397a85 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:46.857937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ae20d-93fa-47f5-8da2-993c86397a85 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:46.863357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fba54f-96ac-4484-800f-d66bd7a064c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:46.864669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fba54f-96ac-4484-800f-d66bd7a064c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.865147 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fba54f-96ac-4484-800f-d66bd7a064c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.865625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fba54f-96ac-4484-800f-d66bd7a064c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.866982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fba54f-96ac-4484-800f-d66bd7a064c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:46.870019 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fba54f-96ac-4484-800f-d66bd7a064c9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:46.882390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1215/2429] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:08:46.886602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a3c04-609a-471e-95ec-387fc0e13958 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:46.899711 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a3c04-609a-471e-95ec-387fc0e13958 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:46.904462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4aee1-a0a6-413f-b993-aa1243c2da98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:46.905360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4aee1-a0a6-413f-b993-aa1243c2da98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.905643 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4aee1-a0a6-413f-b993-aa1243c2da98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.905923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4aee1-a0a6-413f-b993-aa1243c2da98 admin admin] PATH_INFO: `/v3/projects/b3e5e3a025e949b2a619d2cff954e769/users/58b7a4d61c4246cf8e98c8c421b296f0/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.922193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4aee1-a0a6-413f-b993-aa1243c2da98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e5e3a025e949b2a619d2cff954e769, user_id=58b7a4d61c4246cf8e98c8c421b296f0, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:46.924871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4aee1-a0a6-413f-b993-aa1243c2da98 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:46.953476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1215/2430] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:08:46 2022] PUT /identity/v3/projects/b3e5e3a025e949b2a619d2cff954e769/users/58b7a4d61c4246cf8e98c8c421b296f0/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:46.964808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d1e48-24df-4b5d-ad45-e7baf9a424ae None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:46.965133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d1e48-24df-4b5d-ad45-e7baf9a424ae None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:46.965413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d1e48-24df-4b5d-ad45-e7baf9a424ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:46.994406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3a5d1e48-24df-4b5d-ad45-e7baf9a424ae None None] MFA Rules not processed for user `58b7a4d61c4246cf8e98c8c421b296f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:46.995574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3a5d1e48-24df-4b5d-ad45-e7baf9a424ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:47.071517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1216/2431] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:47.075809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4aee1-a0a6-413f-b993-aa1243c2da98 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:47.142354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4aee1-a0a6-413f-b993-aa1243c2da98 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:47.147413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4dc25-1cc2-4f15-a524-217b0f9c9057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:47.148107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4dc25-1cc2-4f15-a524-217b0f9c9057 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:47.148427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4dc25-1cc2-4f15-a524-217b0f9c9057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:47.148870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4dc25-1cc2-4f15-a524-217b0f9c9057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:47.149606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4dc25-1cc2-4f15-a524-217b0f9c9057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:47.151563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4dc25-1cc2-4f15-a524-217b0f9c9057 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:47.178277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1216/2432] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:08:47 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:08:47.182354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d1e48-24df-4b5d-ad45-e7baf9a424ae None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:47.193822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d1e48-24df-4b5d-ad45-e7baf9a424ae None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:47.251070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477bc1ce-da6d-431f-9ac7-e41892503833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:47.251737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bc1ce-da6d-431f-9ac7-e41892503833 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:47.252017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bc1ce-da6d-431f-9ac7-e41892503833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:47.252317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bc1ce-da6d-431f-9ac7-e41892503833 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:47.253173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477bc1ce-da6d-431f-9ac7-e41892503833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:47.255765 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477bc1ce-da6d-431f-9ac7-e41892503833 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:47.275888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1217/2433] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:08:47 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:08:47.279880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4dc25-1cc2-4f15-a524-217b0f9c9057 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:47.290934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4dc25-1cc2-4f15-a524-217b0f9c9057 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:47.295490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74298c32-fabd-4dbe-a373-b5cf73180133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:47.296151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74298c32-fabd-4dbe-a373-b5cf73180133 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:47.296478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74298c32-fabd-4dbe-a373-b5cf73180133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:47.296756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74298c32-fabd-4dbe-a373-b5cf73180133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:47.297484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74298c32-fabd-4dbe-a373-b5cf73180133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:47.299070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74298c32-fabd-4dbe-a373-b5cf73180133 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:47.310807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1217/2434] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:08:47.315063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477bc1ce-da6d-431f-9ac7-e41892503833 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:47.326003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477bc1ce-da6d-431f-9ac7-e41892503833 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:47.330707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7c29b-a7b2-43df-9bc5-de1a62935db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:47.331581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c29b-a7b2-43df-9bc5-de1a62935db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:47.331861 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c29b-a7b2-43df-9bc5-de1a62935db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:47.332137 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c29b-a7b2-43df-9bc5-de1a62935db7 admin admin] PATH_INFO: `/v3/projects/b3878d5c19cb4e0cbca1caecb5ea0da1/users/3deb85e10ded43f595b3d202b8940987/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:47.350051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7c29b-a7b2-43df-9bc5-de1a62935db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3878d5c19cb4e0cbca1caecb5ea0da1, user_id=3deb85e10ded43f595b3d202b8940987, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:47.352801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7c29b-a7b2-43df-9bc5-de1a62935db7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:47.366303 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1218/2435] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:08:47 2022] PUT /identity/v3/projects/b3878d5c19cb4e0cbca1caecb5ea0da1/users/3deb85e10ded43f595b3d202b8940987/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:47.370317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74298c32-fabd-4dbe-a373-b5cf73180133 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:47.380717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74298c32-fabd-4dbe-a373-b5cf73180133 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:47.437716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63456be0-ae8f-410f-8e84-be4c1bc6b234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:47.438311 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63456be0-ae8f-410f-8e84-be4c1bc6b234 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:47.438597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63456be0-ae8f-410f-8e84-be4c1bc6b234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:47.438888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63456be0-ae8f-410f-8e84-be4c1bc6b234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:47.439603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63456be0-ae8f-410f-8e84-be4c1bc6b234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:47.441234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63456be0-ae8f-410f-8e84-be4c1bc6b234 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:47.450994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1218/2436] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:08:47.455730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7c29b-a7b2-43df-9bc5-de1a62935db7 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:47.467588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7c29b-a7b2-43df-9bc5-de1a62935db7 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:47.472268 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add0b7b-69dc-4249-88a9-b24cf4f07ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:47.473174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add0b7b-69dc-4249-88a9-b24cf4f07ddc admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:47.473958 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add0b7b-69dc-4249-88a9-b24cf4f07ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:47.474887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add0b7b-69dc-4249-88a9-b24cf4f07ddc admin admin] PATH_INFO: `/v3/projects/b3878d5c19cb4e0cbca1caecb5ea0da1/users/3deb85e10ded43f595b3d202b8940987/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:47.476885 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add0b7b-69dc-4249-88a9-b24cf4f07ddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3878d5c19cb4e0cbca1caecb5ea0da1, user_id=3deb85e10ded43f595b3d202b8940987, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:47.479542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add0b7b-69dc-4249-88a9-b24cf4f07ddc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:47.481760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:47.492477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1219/2437] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:08:47 2022] PUT /identity/v3/projects/b3878d5c19cb4e0cbca1caecb5ea0da1/users/3deb85e10ded43f595b3d202b8940987/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:47.503555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2418eb-c5eb-44f5-ae6b-9c0b02652356 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:47.503943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2418eb-c5eb-44f5-ae6b-9c0b02652356 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:47.504285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2418eb-c5eb-44f5-ae6b-9c0b02652356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:47.522621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:47.533249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8a2418eb-c5eb-44f5-ae6b-9c0b02652356 None None] MFA Rules not processed for user `3deb85e10ded43f595b3d202b8940987`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:47.534927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8a2418eb-c5eb-44f5-ae6b-9c0b02652356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:47.618113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1219/2438] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:47.630926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add0b7b-69dc-4249-88a9-b24cf4f07ddc admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:47.632688 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8add0b7b-69dc-4249-88a9-b24cf4f07ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:47.698754 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add0b7b-69dc-4249-88a9-b24cf4f07ddc admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:47.702788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e37560-0d29-494f-aa27-af208d9c849d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:47.703503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e37560-0d29-494f-aa27-af208d9c849d service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:47.703783 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e37560-0d29-494f-aa27-af208d9c849d service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:47.704057 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e37560-0d29-494f-aa27-af208d9c849d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:47.715428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e37560-0d29-494f-aa27-af208d9c849d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:47.716943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e37560-0d29-494f-aa27-af208d9c849d service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:47.727367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1220/2439] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:08:47.800761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2418eb-c5eb-44f5-ae6b-9c0b02652356 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:47.811714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2418eb-c5eb-44f5-ae6b-9c0b02652356 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:47.815918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d2a35-b005-4888-9cab-b4eea1271939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:47.816621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d2a35-b005-4888-9cab-b4eea1271939 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:47.817718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d2a35-b005-4888-9cab-b4eea1271939 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:47.818218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d2a35-b005-4888-9cab-b4eea1271939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:47.879588 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d2a35-b005-4888-9cab-b4eea1271939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:47.882312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d2a35-b005-4888-9cab-b4eea1271939 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:47.892278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1220/2440] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:08:57.722118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:57.887876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=58 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:58.126065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee180e-0c44-4176-9d4c-8fda98b693d1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:58.126354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee180e-0c44-4176-9d4c-8fda98b693d1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:58.126633 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee180e-0c44-4176-9d4c-8fda98b693d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:58.145271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-15ee180e-0c44-4176-9d4c-8fda98b693d1 None None] MFA Rules not processed for user `293d4ca894e24e4fb6c4e7aa14256d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:58.146424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-15ee180e-0c44-4176-9d4c-8fda98b693d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:58.204326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1221/2441] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:58.252942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d2a35-b005-4888-9cab-b4eea1271939 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:58.262764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d2a35-b005-4888-9cab-b4eea1271939 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:58.266555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c90b7-61ec-4fe8-8b66-f80ebae66a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:58.267308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c90b7-61ec-4fe8-8b66-f80ebae66a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:58.267659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c90b7-61ec-4fe8-8b66-f80ebae66a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:58.268009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c90b7-61ec-4fe8-8b66-f80ebae66a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:58.270023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9d3c90b7-61ec-4fe8-8b66-f80ebae66a0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:58.342267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c90b7-61ec-4fe8-8b66-f80ebae66a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:58.343693 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c90b7-61ec-4fe8-8b66-f80ebae66a0a service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:58.355037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1221/2442] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:08:58.759829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee180e-0c44-4176-9d4c-8fda98b693d1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:58.821635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee180e-0c44-4176-9d4c-8fda98b693d1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:58.826092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:58.826952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:58.827228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:58.827498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] PATH_INFO: `/v3/users/58b7a4d61c4246cf8e98c8c421b296f0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:58.828807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58b7a4d61c4246cf8e98c8c421b296f0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:58.830754 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:58.880056 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58b7a4d61c4246cf8e98c8c421b296f0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:58.887404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58b7a4d61c4246cf8e98c8c421b296f0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:58.888098 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58b7a4d61c4246cf8e98c8c421b296f0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:58.900696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58b7a4d61c4246cf8e98c8c421b296f0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:58.912787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58b7a4d61c4246cf8e98c8c421b296f0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:58.927585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1222/2443] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:08:58 2022] DELETE /identity/v3/users/58b7a4d61c4246cf8e98c8c421b296f0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:58.932388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1e73f-48df-4cd7-8542-7e72bb46610e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:58.932917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1e73f-48df-4cd7-8542-7e72bb46610e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:58.933032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1e73f-48df-4cd7-8542-7e72bb46610e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:58.951611 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fad1e73f-48df-4cd7-8542-7e72bb46610e None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:08:58.953058 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fad1e73f-48df-4cd7-8542-7e72bb46610e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:59.020081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1222/2444] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:08:59.027736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:59.029410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:59.059283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:59.092497 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f6c8a-5885-4ea6-bc7b-52c476f8bd10 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:59.097555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94372db-adf0-4df7-b23c-17447d3bf7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:59.099259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94372db-adf0-4df7-b23c-17447d3bf7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:59.099600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94372db-adf0-4df7-b23c-17447d3bf7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:59.100076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94372db-adf0-4df7-b23c-17447d3bf7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:59.110671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94372db-adf0-4df7-b23c-17447d3bf7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:59.112154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94372db-adf0-4df7-b23c-17447d3bf7d3 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:59.123913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1223/2445] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:08:59.134982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1e73f-48df-4cd7-8542-7e72bb46610e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:59.136719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fad1e73f-48df-4cd7-8542-7e72bb46610e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:59.151070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1e73f-48df-4cd7-8542-7e72bb46610e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:59.156156 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf011b-8773-4d2c-bc28-42a18ded78bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:59.156950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf011b-8773-4d2c-bc28-42a18ded78bc service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:59.157439 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf011b-8773-4d2c-bc28-42a18ded78bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:59.157795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf011b-8773-4d2c-bc28-42a18ded78bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:59.160022 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0daf011b-8773-4d2c-bc28-42a18ded78bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:59.255142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf011b-8773-4d2c-bc28-42a18ded78bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:59.256958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf011b-8773-4d2c-bc28-42a18ded78bc service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:59.277214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1223/2446] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:08:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:08:59.464674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94372db-adf0-4df7-b23c-17447d3bf7d3 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:59.466352 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f94372db-adf0-4df7-b23c-17447d3bf7d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:08:59.485588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94372db-adf0-4df7-b23c-17447d3bf7d3 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:59.490432 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:59.491205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:59.491815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:59.492091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] PATH_INFO: `/v3/projects/b3e5e3a025e949b2a619d2cff954e769` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:59.495099 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e5e3a025e949b2a619d2cff954e769)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:59.497278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:59.549320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e5e3a025e949b2a619d2cff954e769', 'request_id': 'req-a2b8589d-5361-4b54-82e3-b15bd997370d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:59.559072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e5e3a025e949b2a619d2cff954e769', 'request_id': 'req-a2b8589d-5361-4b54-82e3-b15bd997370d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:59.571979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e5e3a025e949b2a619d2cff954e769', 'request_id': 'req-a2b8589d-5361-4b54-82e3-b15bd997370d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:59.583128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] The token cache is being invalidate because project b3e5e3a025e949b2a619d2cff954e769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:08:59.583419 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:59.584250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:59.586039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1224/2447] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:08:59 2022] DELETE /identity/v3/projects/b3e5e3a025e949b2a619d2cff954e769 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:08:59.590336 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf011b-8773-4d2c-bc28-42a18ded78bc service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:08:59.603739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf011b-8773-4d2c-bc28-42a18ded78bc service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:08:59.611886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:08:59.664129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:08:59.665046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:08:59.665330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:08:59.665605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] PATH_INFO: `/v3/users/3deb85e10ded43f595b3d202b8940987` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:08:59.666786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3deb85e10ded43f595b3d202b8940987)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:08:59.668800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:08:59.714910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3deb85e10ded43f595b3d202b8940987'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:59.715562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3deb85e10ded43f595b3d202b8940987'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:59.716320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3deb85e10ded43f595b3d202b8940987'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:59.724346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3deb85e10ded43f595b3d202b8940987'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:59.737875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-72291ce4-ca9c-4ce3-ab61-3037a440facc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3deb85e10ded43f595b3d202b8940987'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:08:59.755905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1224/2448] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:08:59 2022] DELETE /identity/v3/users/3deb85e10ded43f595b3d202b8940987 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:00.024856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:00.026342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:00.040651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8589d-5361-4b54-82e3-b15bd997370d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:00.108267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:00.109254 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.109545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.109822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] PATH_INFO: `/v3/projects/b3878d5c19cb4e0cbca1caecb5ea0da1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.111010 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3878d5c19cb4e0cbca1caecb5ea0da1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:00.113212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:00.174415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3878d5c19cb4e0cbca1caecb5ea0da1', 'request_id': 'req-6289405d-b852-49bc-b540-2b5d7dcaac7f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:00.177580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:00.183298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3878d5c19cb4e0cbca1caecb5ea0da1', 'request_id': 'req-6289405d-b852-49bc-b540-2b5d7dcaac7f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:00.197758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3878d5c19cb4e0cbca1caecb5ea0da1', 'request_id': 'req-6289405d-b852-49bc-b540-2b5d7dcaac7f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:00.214065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] The token cache is being invalidate because project b3878d5c19cb4e0cbca1caecb5ea0da1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:09:00.214362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:00.215140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6289405d-b852-49bc-b540-2b5d7dcaac7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:00.216936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1225/2449] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:09:00 2022] DELETE /identity/v3/projects/b3878d5c19cb4e0cbca1caecb5ea0da1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:09:00.222237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957a046-e018-43d5-b0b2-2e11888fbdc1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.223161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957a046-e018-43d5-b0b2-2e11888fbdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.223503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957a046-e018-43d5-b0b2-2e11888fbdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.244282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a957a046-e018-43d5-b0b2-2e11888fbdc1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:00.245944 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a957a046-e018-43d5-b0b2-2e11888fbdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:00.315089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1225/2450] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:09:00.323172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed638b1-0068-4b2a-b147-3b2f7c145bb0 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.323488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed638b1-0068-4b2a-b147-3b2f7c145bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.323771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed638b1-0068-4b2a-b147-3b2f7c145bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.356162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1ed638b1-0068-4b2a-b147-3b2f7c145bb0 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:00.359201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1ed638b1-0068-4b2a-b147-3b2f7c145bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:00.368650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1226/2451] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:00.372582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957a046-e018-43d5-b0b2-2e11888fbdc1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:00.382937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957a046-e018-43d5-b0b2-2e11888fbdc1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:00.388358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ae4a4-f40c-4626-8da6-f6eac45ee4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:00.389143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ae4a4-f40c-4626-8da6-f6eac45ee4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.389429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ae4a4-f40c-4626-8da6-f6eac45ee4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.389705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ae4a4-f40c-4626-8da6-f6eac45ee4a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.390438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ae4a4-f40c-4626-8da6-f6eac45ee4a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:09:00.390743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ae4a4-f40c-4626-8da6-f6eac45ee4a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:00.392726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ae4a4-f40c-4626-8da6-f6eac45ee4a7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:00.408338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1226/2452] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 18:09:00.412868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed638b1-0068-4b2a-b147-3b2f7c145bb0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:00.423984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed638b1-0068-4b2a-b147-3b2f7c145bb0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:00.430062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98506aa-7040-4e81-a3ee-674b320f1131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:00.431210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98506aa-7040-4e81-a3ee-674b320f1131 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.431487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98506aa-7040-4e81-a3ee-674b320f1131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.431821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98506aa-7040-4e81-a3ee-674b320f1131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.432551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98506aa-7040-4e81-a3ee-674b320f1131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:00.434573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98506aa-7040-4e81-a3ee-674b320f1131 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:00.463917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1227/2453] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:00 2022] POST /identity/v3/projects => generated 351 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:00.468314 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ae4a4-f40c-4626-8da6-f6eac45ee4a7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:00.479169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ae4a4-f40c-4626-8da6-f6eac45ee4a7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:00.497838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:00.547178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fbff9-d718-4272-88b7-d0dde1635e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:00.547854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fbff9-d718-4272-88b7-d0dde1635e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.548136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fbff9-d718-4272-88b7-d0dde1635e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.548444 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fbff9-d718-4272-88b7-d0dde1635e52 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.549132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fbff9-d718-4272-88b7-d0dde1635e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:00.551110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fbff9-d718-4272-88b7-d0dde1635e52 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:00.574460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1227/2454] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:00 2022] POST /identity/v3/users => generated 276 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:00.576156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98506aa-7040-4e81-a3ee-674b320f1131 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:00.588254 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98506aa-7040-4e81-a3ee-674b320f1131 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:00.593538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7174943-ca4c-494c-a44f-59906859957b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:00.594210 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7174943-ca4c-494c-a44f-59906859957b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.594495 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7174943-ca4c-494c-a44f-59906859957b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.594788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7174943-ca4c-494c-a44f-59906859957b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.595509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7174943-ca4c-494c-a44f-59906859957b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:00.597190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7174943-ca4c-494c-a44f-59906859957b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:00.609506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1228/2455] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:00.613645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fbff9-d718-4272-88b7-d0dde1635e52 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:00.623649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fbff9-d718-4272-88b7-d0dde1635e52 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:00.629045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c67b4-abc4-4ebf-88fb-cf8ab6b720f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:00.629973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c67b4-abc4-4ebf-88fb-cf8ab6b720f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.630283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c67b4-abc4-4ebf-88fb-cf8ab6b720f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.630559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c67b4-abc4-4ebf-88fb-cf8ab6b720f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.631318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c67b4-abc4-4ebf-88fb-cf8ab6b720f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:00.632953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c67b4-abc4-4ebf-88fb-cf8ab6b720f0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:00.644993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1228/2456] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:00.649691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7174943-ca4c-494c-a44f-59906859957b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:00.661399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7174943-ca4c-494c-a44f-59906859957b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:00.666542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd125feb-01af-4a21-8e77-a07d3da27d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:00.667715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd125feb-01af-4a21-8e77-a07d3da27d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.668066 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd125feb-01af-4a21-8e77-a07d3da27d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.668445 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd125feb-01af-4a21-8e77-a07d3da27d86 admin admin] PATH_INFO: `/v3/projects/8afc8e4291e6427dbae17b80c6b41f76/users/f1bf26c2b8f64d4785bc51dc149adfa6/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.684722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd125feb-01af-4a21-8e77-a07d3da27d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afc8e4291e6427dbae17b80c6b41f76, user_id=f1bf26c2b8f64d4785bc51dc149adfa6, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:00.687377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd125feb-01af-4a21-8e77-a07d3da27d86 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:00.701184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1229/2457] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:00 2022] PUT /identity/v3/projects/8afc8e4291e6427dbae17b80c6b41f76/users/f1bf26c2b8f64d4785bc51dc149adfa6/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:00.712956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842074ab-bb99-4633-9b13-fc5bec354d96 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.713333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842074ab-bb99-4633-9b13-fc5bec354d96 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.713564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842074ab-bb99-4633-9b13-fc5bec354d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.741981 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-842074ab-bb99-4633-9b13-fc5bec354d96 None None] MFA Rules not processed for user `f1bf26c2b8f64d4785bc51dc149adfa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:00.743274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-842074ab-bb99-4633-9b13-fc5bec354d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:00.821022 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1229/2458] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:00 2022] POST /identity/v3/auth/tokens => generated 3218 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:00.829112 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd125feb-01af-4a21-8e77-a07d3da27d86 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:00.899875 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd125feb-01af-4a21-8e77-a07d3da27d86 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:00.904945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9115d5-5190-4927-97bc-95e43b3d000b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:00.905649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9115d5-5190-4927-97bc-95e43b3d000b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:00.905930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9115d5-5190-4927-97bc-95e43b3d000b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:00.906209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9115d5-5190-4927-97bc-95e43b3d000b admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:00.906940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9115d5-5190-4927-97bc-95e43b3d000b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:00.908845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9115d5-5190-4927-97bc-95e43b3d000b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:00.943637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1230/2459] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:00 2022] POST /identity/v3/projects => generated 353 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:00.948714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842074ab-bb99-4633-9b13-fc5bec354d96 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:00.963757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842074ab-bb99-4633-9b13-fc5bec354d96 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:01.072544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c28ea-6a47-4e9a-a2c1-1b3206a38910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:01.073180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c28ea-6a47-4e9a-a2c1-1b3206a38910 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:01.073468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c28ea-6a47-4e9a-a2c1-1b3206a38910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:01.073752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c28ea-6a47-4e9a-a2c1-1b3206a38910 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:01.074453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c28ea-6a47-4e9a-a2c1-1b3206a38910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:01.077236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c28ea-6a47-4e9a-a2c1-1b3206a38910 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:01.107900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1230/2460] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:00 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:01.111660 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9115d5-5190-4927-97bc-95e43b3d000b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:01.122805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9115d5-5190-4927-97bc-95e43b3d000b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:01.127322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad35ef0-e118-49da-b08c-6b684d0e457b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:01.127979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad35ef0-e118-49da-b08c-6b684d0e457b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:01.128258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad35ef0-e118-49da-b08c-6b684d0e457b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:01.128563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad35ef0-e118-49da-b08c-6b684d0e457b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:01.129323 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad35ef0-e118-49da-b08c-6b684d0e457b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:01.130953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad35ef0-e118-49da-b08c-6b684d0e457b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:01.142591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1231/2461] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 18:09:01.147429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c28ea-6a47-4e9a-a2c1-1b3206a38910 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:01.157265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c28ea-6a47-4e9a-a2c1-1b3206a38910 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:01.162721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f58f15-f5ed-49b7-bd0a-be9e5d22fd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:01.163597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f58f15-f5ed-49b7-bd0a-be9e5d22fd98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:01.163880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f58f15-f5ed-49b7-bd0a-be9e5d22fd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:01.164153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f58f15-f5ed-49b7-bd0a-be9e5d22fd98 admin admin] PATH_INFO: `/v3/projects/50bd1ff063fa43e0ba4a9e1bf5f13edf/users/bd5d08a3a3e74ca79a5810230e1a6f17/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:01.177205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:01.180205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f58f15-f5ed-49b7-bd0a-be9e5d22fd98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50bd1ff063fa43e0ba4a9e1bf5f13edf, user_id=bd5d08a3a3e74ca79a5810230e1a6f17, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:01.183348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f58f15-f5ed-49b7-bd0a-be9e5d22fd98 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:01.200476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1231/2462] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:01 2022] PUT /identity/v3/projects/50bd1ff063fa43e0ba4a9e1bf5f13edf/users/bd5d08a3a3e74ca79a5810230e1a6f17/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:01.204680 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad35ef0-e118-49da-b08c-6b684d0e457b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:01.215551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad35ef0-e118-49da-b08c-6b684d0e457b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:01.274830 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:01.289791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdc032-727d-4712-88ae-356775ed7b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:01.291076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdc032-727d-4712-88ae-356775ed7b55 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:01.291653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdc032-727d-4712-88ae-356775ed7b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:01.292344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdc032-727d-4712-88ae-356775ed7b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:01.293162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdc032-727d-4712-88ae-356775ed7b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:01.295769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdc032-727d-4712-88ae-356775ed7b55 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:01.309079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1232/2463] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:01.313038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f58f15-f5ed-49b7-bd0a-be9e5d22fd98 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:01.324749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f58f15-f5ed-49b7-bd0a-be9e5d22fd98 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:01.330383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be1523-b478-4c59-97ce-f48fd2f08d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:01.331274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be1523-b478-4c59-97ce-f48fd2f08d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:01.331562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be1523-b478-4c59-97ce-f48fd2f08d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:01.332508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be1523-b478-4c59-97ce-f48fd2f08d9e admin admin] PATH_INFO: `/v3/projects/50bd1ff063fa43e0ba4a9e1bf5f13edf/users/bd5d08a3a3e74ca79a5810230e1a6f17/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:01.334521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be1523-b478-4c59-97ce-f48fd2f08d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50bd1ff063fa43e0ba4a9e1bf5f13edf, user_id=bd5d08a3a3e74ca79a5810230e1a6f17, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:01.337222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be1523-b478-4c59-97ce-f48fd2f08d9e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:01.349987 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1232/2464] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:01 2022] PUT /identity/v3/projects/50bd1ff063fa43e0ba4a9e1bf5f13edf/users/bd5d08a3a3e74ca79a5810230e1a6f17/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:09:01.361316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451365b4-d742-4d12-9f09-92b3213b34db None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:01.361752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451365b4-d742-4d12-9f09-92b3213b34db None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:01.362464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451365b4-d742-4d12-9f09-92b3213b34db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:01.404490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-451365b4-d742-4d12-9f09-92b3213b34db None None] MFA Rules not processed for user `bd5d08a3a3e74ca79a5810230e1a6f17`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:01.405857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-451365b4-d742-4d12-9f09-92b3213b34db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:01.497976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1233/2465] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:01 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:01.508654 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be1523-b478-4c59-97ce-f48fd2f08d9e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:01.514653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-94be1523-b478-4c59-97ce-f48fd2f08d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:01.595389 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be1523-b478-4c59-97ce-f48fd2f08d9e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:01.600771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e6fa3-9c28-46b8-bcfd-7d0c0e7afd9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:01.601660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e6fa3-9c28-46b8-bcfd-7d0c0e7afd9f service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:01.602134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e6fa3-9c28-46b8-bcfd-7d0c0e7afd9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:01.602600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e6fa3-9c28-46b8-bcfd-7d0c0e7afd9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:01.613027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e6fa3-9c28-46b8-bcfd-7d0c0e7afd9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:01.615290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e6fa3-9c28-46b8-bcfd-7d0c0e7afd9f service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:01.626999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1233/2466] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:01 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:01.720597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451365b4-d742-4d12-9f09-92b3213b34db None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:01.732361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451365b4-d742-4d12-9f09-92b3213b34db None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:01.737346 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa5492-8b09-48b6-8529-806231897cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:01.738262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa5492-8b09-48b6-8529-806231897cbc service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:01.738558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa5492-8b09-48b6-8529-806231897cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:01.738835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa5492-8b09-48b6-8529-806231897cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:01.831702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa5492-8b09-48b6-8529-806231897cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:01.833335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa5492-8b09-48b6-8529-806231897cbc service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:01.844648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1234/2467] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:01 2022] GET /identity/v3/auth/tokens => generated 3218 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:03.654272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e6fa3-9c28-46b8-bcfd-7d0c0e7afd9f service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:03.655653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b09e6fa3-9c28-46b8-bcfd-7d0c0e7afd9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:03.712096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e6fa3-9c28-46b8-bcfd-7d0c0e7afd9f service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:03.715655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e97d3-d80d-44c3-92b1-42e8fea35196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:03.716286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e97d3-d80d-44c3-92b1-42e8fea35196 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:03.716594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e97d3-d80d-44c3-92b1-42e8fea35196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:03.716868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e97d3-d80d-44c3-92b1-42e8fea35196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:03.718600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-172e97d3-d80d-44c3-92b1-42e8fea35196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:03.781771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e97d3-d80d-44c3-92b1-42e8fea35196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:03.783146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e97d3-d80d-44c3-92b1-42e8fea35196 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:03.794339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1234/2468] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:07.019114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa5492-8b09-48b6-8529-806231897cbc service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:07.021314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-31fa5492-8b09-48b6-8529-806231897cbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:07.041517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa5492-8b09-48b6-8529-806231897cbc service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:07.046197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:07.047237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.047525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.047805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] PATH_INFO: `/v3/users/fa82bddf9aa0455797d42993fe850ccf` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.049020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa82bddf9aa0455797d42993fe850ccf)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:07.051850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:07.104894 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa82bddf9aa0455797d42993fe850ccf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:07.112387 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa82bddf9aa0455797d42993fe850ccf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:07.114603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa82bddf9aa0455797d42993fe850ccf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:07.121873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:07.128145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa82bddf9aa0455797d42993fe850ccf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:07.136802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-04e96bc4-b2c1-4d46-8174-85a0d3f920e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa82bddf9aa0455797d42993fe850ccf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:07.146499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1235/2469] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:09:07 2022] DELETE /identity/v3/users/fa82bddf9aa0455797d42993fe850ccf => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:07.321119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e97d3-d80d-44c3-92b1-42e8fea35196 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:07.333279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e97d3-d80d-44c3-92b1-42e8fea35196 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:07.373199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:07.404869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:07.405786 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.406080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.406409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] PATH_INFO: `/v3/projects/b0917b2a75df4a9ab1d2cbd4fd3f7613` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.407612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0917b2a75df4a9ab1d2cbd4fd3f7613)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:07.409575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:07.460571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0917b2a75df4a9ab1d2cbd4fd3f7613', 'request_id': 'req-80c40db7-031a-4cf4-9678-d66ed3f8a83a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:07.468532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0917b2a75df4a9ab1d2cbd4fd3f7613', 'request_id': 'req-80c40db7-031a-4cf4-9678-d66ed3f8a83a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:07.475049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0917b2a75df4a9ab1d2cbd4fd3f7613', 'request_id': 'req-80c40db7-031a-4cf4-9678-d66ed3f8a83a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:07.486185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] The token cache is being invalidate because project b0917b2a75df4a9ab1d2cbd4fd3f7613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:09:07.486477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:07.487150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-80c40db7-031a-4cf4-9678-d66ed3f8a83a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:07.488882 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1235/2470] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:09:07 2022] DELETE /identity/v3/projects/b0917b2a75df4a9ab1d2cbd4fd3f7613 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:07.495544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35637823-ac84-4aaf-a507-7339d7920ec7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.495907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35637823-ac84-4aaf-a507-7339d7920ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.496274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35637823-ac84-4aaf-a507-7339d7920ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.514966 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-35637823-ac84-4aaf-a507-7339d7920ec7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:07.516217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-35637823-ac84-4aaf-a507-7339d7920ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:07.578083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1236/2471] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:07.586405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341c9a1-eb61-4a5f-96c9-209ddc5327da None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.586697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341c9a1-eb61-4a5f-96c9-209ddc5327da None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.586996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341c9a1-eb61-4a5f-96c9-209ddc5327da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.605461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0341c9a1-eb61-4a5f-96c9-209ddc5327da None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:07.606658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0341c9a1-eb61-4a5f-96c9-209ddc5327da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:07.615089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1236/2472] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:07.619133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35637823-ac84-4aaf-a507-7339d7920ec7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:07.629690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35637823-ac84-4aaf-a507-7339d7920ec7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:07.634220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cd1d8-0567-4532-9c3b-3c9de4974df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:07.634872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cd1d8-0567-4532-9c3b-3c9de4974df4 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.635158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cd1d8-0567-4532-9c3b-3c9de4974df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.635439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cd1d8-0567-4532-9c3b-3c9de4974df4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.636164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cd1d8-0567-4532-9c3b-3c9de4974df4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:09:07.636501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cd1d8-0567-4532-9c3b-3c9de4974df4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:07.638111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cd1d8-0567-4532-9c3b-3c9de4974df4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:07.652357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1237/2473] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:09:07.654323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341c9a1-eb61-4a5f-96c9-209ddc5327da None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:07.663577 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341c9a1-eb61-4a5f-96c9-209ddc5327da None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:07.668222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d6f28-10c8-4dec-b571-788c3df2c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:07.668908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d6f28-10c8-4dec-b571-788c3df2c1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.669214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d6f28-10c8-4dec-b571-788c3df2c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.669494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d6f28-10c8-4dec-b571-788c3df2c1a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.670183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d6f28-10c8-4dec-b571-788c3df2c1a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:07.672074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d6f28-10c8-4dec-b571-788c3df2c1a8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:07.697750 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1237/2474] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:07 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:07.701720 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cd1d8-0567-4532-9c3b-3c9de4974df4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:07.712025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cd1d8-0567-4532-9c3b-3c9de4974df4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:07.770554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2ca8-7a7d-498e-a2d1-8560d9569bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:07.771230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2ca8-7a7d-498e-a2d1-8560d9569bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.771512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2ca8-7a7d-498e-a2d1-8560d9569bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.771816 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2ca8-7a7d-498e-a2d1-8560d9569bb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.772551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe2ca8-7a7d-498e-a2d1-8560d9569bb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:07.774451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe2ca8-7a7d-498e-a2d1-8560d9569bb3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:07.796531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1238/2475] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:07 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:07.797746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d6f28-10c8-4dec-b571-788c3df2c1a8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:07.808626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d6f28-10c8-4dec-b571-788c3df2c1a8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:07.813179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8776208-6722-4232-8880-6ad13cd43a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:07.813843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8776208-6722-4232-8880-6ad13cd43a37 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.814130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8776208-6722-4232-8880-6ad13cd43a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.814408 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8776208-6722-4232-8880-6ad13cd43a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.815136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8776208-6722-4232-8880-6ad13cd43a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:07.817458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8776208-6722-4232-8880-6ad13cd43a37 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:07.827818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1238/2476] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:07.831428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2ca8-7a7d-498e-a2d1-8560d9569bb3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:07.841957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2ca8-7a7d-498e-a2d1-8560d9569bb3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:07.846418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb084a23-231e-4fc4-9c66-347ace4660a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:07.847061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb084a23-231e-4fc4-9c66-347ace4660a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.847844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb084a23-231e-4fc4-9c66-347ace4660a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.848117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb084a23-231e-4fc4-9c66-347ace4660a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.848940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb084a23-231e-4fc4-9c66-347ace4660a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:07.850873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb084a23-231e-4fc4-9c66-347ace4660a4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:07.862620 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1239/2477] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:07.867572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8776208-6722-4232-8880-6ad13cd43a37 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:07.877546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8776208-6722-4232-8880-6ad13cd43a37 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:07.882646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c094e-6998-43ba-8c47-fc3ba4a21c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:07.883523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c094e-6998-43ba-8c47-fc3ba4a21c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.883805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c094e-6998-43ba-8c47-fc3ba4a21c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.884083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c094e-6998-43ba-8c47-fc3ba4a21c31 admin admin] PATH_INFO: `/v3/projects/c1a13a032ddb4e90a889b9e82c3fbd37/users/d1c3026447e0439c9aa4d40541418ecb/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.899296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c094e-6998-43ba-8c47-fc3ba4a21c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a13a032ddb4e90a889b9e82c3fbd37, user_id=d1c3026447e0439c9aa4d40541418ecb, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:07.901971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c094e-6998-43ba-8c47-fc3ba4a21c31 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:07.916040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1239/2478] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:07 2022] PUT /identity/v3/projects/c1a13a032ddb4e90a889b9e82c3fbd37/users/d1c3026447e0439c9aa4d40541418ecb/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:07.926836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2638-470c-4431-b52d-abba4795f6b7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:07.927128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2638-470c-4431-b52d-abba4795f6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:07.927408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2638-470c-4431-b52d-abba4795f6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:07.954316 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3a8c2638-470c-4431-b52d-abba4795f6b7 None None] MFA Rules not processed for user `d1c3026447e0439c9aa4d40541418ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:07.955528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3a8c2638-470c-4431-b52d-abba4795f6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:07.993754 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:08.021921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1240/2479] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:09:08.032835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c094e-6998-43ba-8c47-fc3ba4a21c31 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:08.034550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-579c094e-6998-43ba-8c47-fc3ba4a21c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:08.092980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:08.112526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c094e-6998-43ba-8c47-fc3ba4a21c31 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:08.118136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e0cb2-dfeb-4c6a-b144-126405f39c9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:08.118812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e0cb2-dfeb-4c6a-b144-126405f39c9b service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:08.119122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e0cb2-dfeb-4c6a-b144-126405f39c9b service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:08.119428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e0cb2-dfeb-4c6a-b144-126405f39c9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:08.127232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e0cb2-dfeb-4c6a-b144-126405f39c9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:08.129311 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e0cb2-dfeb-4c6a-b144-126405f39c9b service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:08.138048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1240/2480] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:15.478986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c2638-470c-4431-b52d-abba4795f6b7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:15.539287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c2638-470c-4431-b52d-abba4795f6b7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:15.543866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:15.544776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:15.545061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:15.545343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] PATH_INFO: `/v3/users/d1c3026447e0439c9aa4d40541418ecb` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:15.546547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c3026447e0439c9aa4d40541418ecb)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:15.548529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:15.594703 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c3026447e0439c9aa4d40541418ecb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:15.602902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c3026447e0439c9aa4d40541418ecb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:15.603573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c3026447e0439c9aa4d40541418ecb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:15.615276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c3026447e0439c9aa4d40541418ecb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:15.622456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c3026447e0439c9aa4d40541418ecb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:15.628362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1241/2481] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:09:15 2022] DELETE /identity/v3/users/d1c3026447e0439c9aa4d40541418ecb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:09:15.632893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef4b92-d3dd-47b7-9cd4-f0e51754e89c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:15.633362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef4b92-d3dd-47b7-9cd4-f0e51754e89c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:15.633487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef4b92-d3dd-47b7-9cd4-f0e51754e89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:15.651132 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a6ef4b92-d3dd-47b7-9cd4-f0e51754e89c None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:15.652381 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a6ef4b92-d3dd-47b7-9cd4-f0e51754e89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:15.714774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1241/2482] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:15.725061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:15.726547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:15.783925 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72e307-6a1f-4a69-a083-3eddc0eb6229 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:15.788404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc936b-9efa-46d9-8c42-7690c448a492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:15.789092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc936b-9efa-46d9-8c42-7690c448a492 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:15.789376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc936b-9efa-46d9-8c42-7690c448a492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:15.789815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc936b-9efa-46d9-8c42-7690c448a492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:15.799219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc936b-9efa-46d9-8c42-7690c448a492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:15.801557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc936b-9efa-46d9-8c42-7690c448a492 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:15.806530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:15.811807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1242/2483] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:16.096631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef4b92-d3dd-47b7-9cd4-f0e51754e89c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:16.098291 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a6ef4b92-d3dd-47b7-9cd4-f0e51754e89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:16.120191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef4b92-d3dd-47b7-9cd4-f0e51754e89c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:16.120601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:16.121477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.121890 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.122290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] PATH_INFO: `/v3/projects/c1a13a032ddb4e90a889b9e82c3fbd37` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.123520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a13a032ddb4e90a889b9e82c3fbd37)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:16.125468 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:16.179743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a13a032ddb4e90a889b9e82c3fbd37', 'request_id': 'req-de690177-62c5-494c-9546-eb88319c30ee'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:16.186578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a13a032ddb4e90a889b9e82c3fbd37', 'request_id': 'req-de690177-62c5-494c-9546-eb88319c30ee'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:16.193627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a13a032ddb4e90a889b9e82c3fbd37', 'request_id': 'req-de690177-62c5-494c-9546-eb88319c30ee'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:16.205364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] The token cache is being invalidate because project c1a13a032ddb4e90a889b9e82c3fbd37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:09:16.205479 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:16.206170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-de690177-62c5-494c-9546-eb88319c30ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:16.208600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1242/2484] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:09:16 2022] DELETE /identity/v3/projects/c1a13a032ddb4e90a889b9e82c3fbd37 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:09:16.214405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb80be-967b-465d-94dc-f629489e76c9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.214807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb80be-967b-465d-94dc-f629489e76c9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.215156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb80be-967b-465d-94dc-f629489e76c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.240047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1ecb80be-967b-465d-94dc-f629489e76c9 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:16.241553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1ecb80be-967b-465d-94dc-f629489e76c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:16.312497 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1243/2485] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:16.320369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e862ec-0c5d-4d54-a7c5-b5a6b32ac6a4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.320777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e862ec-0c5d-4d54-a7c5-b5a6b32ac6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.321062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e862ec-0c5d-4d54-a7c5-b5a6b32ac6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.342180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a0e862ec-0c5d-4d54-a7c5-b5a6b32ac6a4 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:16.343375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a0e862ec-0c5d-4d54-a7c5-b5a6b32ac6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:16.351943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1243/2486] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:16.356096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb80be-967b-465d-94dc-f629489e76c9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:16.366244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb80be-967b-465d-94dc-f629489e76c9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:16.370863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cde4a5-c50a-4dd2-9137-1e600001608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:16.371531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cde4a5-c50a-4dd2-9137-1e600001608e admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.371807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cde4a5-c50a-4dd2-9137-1e600001608e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.372101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cde4a5-c50a-4dd2-9137-1e600001608e admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.372948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cde4a5-c50a-4dd2-9137-1e600001608e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:09:16.373258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cde4a5-c50a-4dd2-9137-1e600001608e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:16.374882 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cde4a5-c50a-4dd2-9137-1e600001608e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:16.386647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1244/2487] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 18:09:16.390847 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e862ec-0c5d-4d54-a7c5-b5a6b32ac6a4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:16.400544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e862ec-0c5d-4d54-a7c5-b5a6b32ac6a4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:16.405227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ae7ba-1948-4eda-bba4-80c16cfbc9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:16.405878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ae7ba-1948-4eda-bba4-80c16cfbc9cf admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.406157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ae7ba-1948-4eda-bba4-80c16cfbc9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.406432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ae7ba-1948-4eda-bba4-80c16cfbc9cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.407109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ae7ba-1948-4eda-bba4-80c16cfbc9cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:16.409059 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ae7ba-1948-4eda-bba4-80c16cfbc9cf admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:16.437859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1244/2488] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:16 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:16.442023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cde4a5-c50a-4dd2-9137-1e600001608e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:16.452404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cde4a5-c50a-4dd2-9137-1e600001608e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:16.508862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb87eca-fc70-4b27-9aec-e19a302dedcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:16.509559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb87eca-fc70-4b27-9aec-e19a302dedcb admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.509841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb87eca-fc70-4b27-9aec-e19a302dedcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.510118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb87eca-fc70-4b27-9aec-e19a302dedcb admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.510807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb87eca-fc70-4b27-9aec-e19a302dedcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:16.512791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb87eca-fc70-4b27-9aec-e19a302dedcb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:16.533709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1245/2489] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:16 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:16.538376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ae7ba-1948-4eda-bba4-80c16cfbc9cf admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:16.548796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ae7ba-1948-4eda-bba4-80c16cfbc9cf admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:16.554099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0b51b-edca-408b-806e-5461e2746d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:16.554759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0b51b-edca-408b-806e-5461e2746d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.555038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0b51b-edca-408b-806e-5461e2746d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.555313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0b51b-edca-408b-806e-5461e2746d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.556034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0b51b-edca-408b-806e-5461e2746d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:16.557703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0b51b-edca-408b-806e-5461e2746d93 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:16.570337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1245/2490] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:16.574378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb87eca-fc70-4b27-9aec-e19a302dedcb admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:16.585337 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb87eca-fc70-4b27-9aec-e19a302dedcb admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:16.589912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04759801-06ea-48a6-9e6c-67bf16fd1479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:16.590563 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04759801-06ea-48a6-9e6c-67bf16fd1479 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.590846 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04759801-06ea-48a6-9e6c-67bf16fd1479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.591118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04759801-06ea-48a6-9e6c-67bf16fd1479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.591835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04759801-06ea-48a6-9e6c-67bf16fd1479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:16.593480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04759801-06ea-48a6-9e6c-67bf16fd1479 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:16.603880 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1246/2491] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:16.608026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0b51b-edca-408b-806e-5461e2746d93 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:16.618559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0b51b-edca-408b-806e-5461e2746d93 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:16.623582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebe548-a3cd-4414-944a-23aa02276fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:16.624633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebe548-a3cd-4414-944a-23aa02276fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.624913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebe548-a3cd-4414-944a-23aa02276fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.625186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebe548-a3cd-4414-944a-23aa02276fe7 admin admin] PATH_INFO: `/v3/projects/fd208bc38bbb4dea88bf582fb1c11e57/users/88851bbc8bf742199ad18572d9a28223/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.640029 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebe548-a3cd-4414-944a-23aa02276fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd208bc38bbb4dea88bf582fb1c11e57, user_id=88851bbc8bf742199ad18572d9a28223, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:16.642703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebe548-a3cd-4414-944a-23aa02276fe7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:16.655180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1246/2492] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:16 2022] PUT /identity/v3/projects/fd208bc38bbb4dea88bf582fb1c11e57/users/88851bbc8bf742199ad18572d9a28223/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:16.711996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ed3d6-9298-4acd-a0f5-705c298a4e7c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.712215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ed3d6-9298-4acd-a0f5-705c298a4e7c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.712490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ed3d6-9298-4acd-a0f5-705c298a4e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.739391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f16ed3d6-9298-4acd-a0f5-705c298a4e7c None None] MFA Rules not processed for user `88851bbc8bf742199ad18572d9a28223`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:16.740624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f16ed3d6-9298-4acd-a0f5-705c298a4e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:16.793442 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:16.804524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1247/2493] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:16.808975 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebe548-a3cd-4414-944a-23aa02276fe7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:16.813548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:16.884098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebe548-a3cd-4414-944a-23aa02276fe7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:16.889799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5629b1-a707-4a58-91e6-2d939e044912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:16.890478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5629b1-a707-4a58-91e6-2d939e044912 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:16.890764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5629b1-a707-4a58-91e6-2d939e044912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:16.891192 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5629b1-a707-4a58-91e6-2d939e044912 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:16.891738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5629b1-a707-4a58-91e6-2d939e044912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:16.894296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5629b1-a707-4a58-91e6-2d939e044912 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:16.927072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1247/2494] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:16 2022] POST /identity/v3/projects => generated 403 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:16.928320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ed3d6-9298-4acd-a0f5-705c298a4e7c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:16.939983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ed3d6-9298-4acd-a0f5-705c298a4e7c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:17.000648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e8412-3777-40ba-8b80-fd466b3dd5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:17.001317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e8412-3777-40ba-8b80-fd466b3dd5aa admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:17.001594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e8412-3777-40ba-8b80-fd466b3dd5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:17.001873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e8412-3777-40ba-8b80-fd466b3dd5aa admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:17.002549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e8412-3777-40ba-8b80-fd466b3dd5aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:17.004555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e8412-3777-40ba-8b80-fd466b3dd5aa admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:17.026866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1248/2495] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:16 2022] POST /identity/v3/users => generated 308 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:17.030653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5629b1-a707-4a58-91e6-2d939e044912 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:17.041603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5629b1-a707-4a58-91e6-2d939e044912 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:17.046749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda67b53-7413-4da3-921a-4ca3d4423530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:17.047415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda67b53-7413-4da3-921a-4ca3d4423530 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:17.047697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda67b53-7413-4da3-921a-4ca3d4423530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:17.047974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda67b53-7413-4da3-921a-4ca3d4423530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:17.048733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda67b53-7413-4da3-921a-4ca3d4423530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:17.050504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda67b53-7413-4da3-921a-4ca3d4423530 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:17.061761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1248/2496] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:17.066204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e8412-3777-40ba-8b80-fd466b3dd5aa admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:17.076568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e8412-3777-40ba-8b80-fd466b3dd5aa admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:17.082002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65998295-582f-4a8d-bef3-54f3f69f2bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:17.083000 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65998295-582f-4a8d-bef3-54f3f69f2bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:17.083357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65998295-582f-4a8d-bef3-54f3f69f2bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:17.083712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65998295-582f-4a8d-bef3-54f3f69f2bd6 admin admin] PATH_INFO: `/v3/projects/2051612041f9440fa1ed21a163a0b5d4/users/3148fe9e3f894182a46c3d1d77c72476/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:17.099971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65998295-582f-4a8d-bef3-54f3f69f2bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2051612041f9440fa1ed21a163a0b5d4, user_id=3148fe9e3f894182a46c3d1d77c72476, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:17.102687 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65998295-582f-4a8d-bef3-54f3f69f2bd6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:17.116393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1249/2497] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:17 2022] PUT /identity/v3/projects/2051612041f9440fa1ed21a163a0b5d4/users/3148fe9e3f894182a46c3d1d77c72476/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:17.120466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda67b53-7413-4da3-921a-4ca3d4423530 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:17.130986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda67b53-7413-4da3-921a-4ca3d4423530 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:17.169108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65998295-582f-4a8d-bef3-54f3f69f2bd6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:17.171249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-65998295-582f-4a8d-bef3-54f3f69f2bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:17.189066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e8db3-db7c-4991-80de-82e684f23806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:17.189737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e8db3-db7c-4991-80de-82e684f23806 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:17.190373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e8db3-db7c-4991-80de-82e684f23806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:17.190373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e8db3-db7c-4991-80de-82e684f23806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:17.191410 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e8db3-db7c-4991-80de-82e684f23806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:17.193033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e8db3-db7c-4991-80de-82e684f23806 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:17.202742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1249/2498] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:17.207489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e8db3-db7c-4991-80de-82e684f23806 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:17.217937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e8db3-db7c-4991-80de-82e684f23806 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:17.223316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084811f-740a-413d-a43f-cb3692533efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:17.224186 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084811f-740a-413d-a43f-cb3692533efc admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:17.224501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084811f-740a-413d-a43f-cb3692533efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:17.224784 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084811f-740a-413d-a43f-cb3692533efc admin admin] PATH_INFO: `/v3/projects/2051612041f9440fa1ed21a163a0b5d4/users/3148fe9e3f894182a46c3d1d77c72476/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:17.226798 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084811f-740a-413d-a43f-cb3692533efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2051612041f9440fa1ed21a163a0b5d4, user_id=3148fe9e3f894182a46c3d1d77c72476, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:17.229477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084811f-740a-413d-a43f-cb3692533efc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:17.230047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65998295-582f-4a8d-bef3-54f3f69f2bd6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:17.234716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f45ef-7287-42a1-8945-ec28deae656f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:17.235353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f45ef-7287-42a1-8945-ec28deae656f service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:17.235635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f45ef-7287-42a1-8945-ec28deae656f service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:17.235912 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f45ef-7287-42a1-8945-ec28deae656f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:17.237928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bc6f45ef-7287-42a1-8945-ec28deae656f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:17.240684 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1250/2499] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:17 2022] PUT /identity/v3/projects/2051612041f9440fa1ed21a163a0b5d4/users/3148fe9e3f894182a46c3d1d77c72476/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:09:17.250568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618b50c-6500-42db-84ee-e8071d7e563f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:17.250852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618b50c-6500-42db-84ee-e8071d7e563f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:17.251126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618b50c-6500-42db-84ee-e8071d7e563f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:17.285725 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f618b50c-6500-42db-84ee-e8071d7e563f None None] MFA Rules not processed for user `3148fe9e3f894182a46c3d1d77c72476`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:17.287368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f618b50c-6500-42db-84ee-e8071d7e563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:17.321181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f45ef-7287-42a1-8945-ec28deae656f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:17.323113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f45ef-7287-42a1-8945-ec28deae656f service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:17.335394 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1250/2500] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:09:17.345560 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:17.372450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1251/2501] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:09:17.387679 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f45ef-7287-42a1-8945-ec28deae656f service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:17.389173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bc6f45ef-7287-42a1-8945-ec28deae656f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:17.404396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:17.457938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f45ef-7287-42a1-8945-ec28deae656f service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:17.512084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f416e00-73c1-4c6a-9d46-3f9dc87d1406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:17.512857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f416e00-73c1-4c6a-9d46-3f9dc87d1406 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:17.513041 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f416e00-73c1-4c6a-9d46-3f9dc87d1406 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:17.513321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f416e00-73c1-4c6a-9d46-3f9dc87d1406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:17.527652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f416e00-73c1-4c6a-9d46-3f9dc87d1406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:17.529267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f416e00-73c1-4c6a-9d46-3f9dc87d1406 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:17.540646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1251/2502] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:17 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:17.714103 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618b50c-6500-42db-84ee-e8071d7e563f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:17.796288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618b50c-6500-42db-84ee-e8071d7e563f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:17.801532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:17.802417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:17.802686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:17.802970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] PATH_INFO: `/v3/users/88851bbc8bf742199ad18572d9a28223` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:17.804249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=88851bbc8bf742199ad18572d9a28223)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:17.806255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:17.851999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88851bbc8bf742199ad18572d9a28223'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:17.852686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88851bbc8bf742199ad18572d9a28223'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:17.853295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88851bbc8bf742199ad18572d9a28223'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:17.861008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88851bbc8bf742199ad18572d9a28223'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:17.873923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88851bbc8bf742199ad18572d9a28223'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:17.889648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1252/2503] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:09:17 2022] DELETE /identity/v3/users/88851bbc8bf742199ad18572d9a28223 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:17.894138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269093c2-5d7e-446f-89ed-0e2de648caeb None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:17.894423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269093c2-5d7e-446f-89ed-0e2de648caeb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:17.894699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269093c2-5d7e-446f-89ed-0e2de648caeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:17.912343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-269093c2-5d7e-446f-89ed-0e2de648caeb None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:17.913976 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-269093c2-5d7e-446f-89ed-0e2de648caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:17.980617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1252/2504] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:09:17.991001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:17.992552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:18.059524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe55bfd-bbc2-4b85-97bd-74b43ffd8a3f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:18.064101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda2884-5e48-4aae-a29b-14edbc75f8c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:18.065442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda2884-5e48-4aae-a29b-14edbc75f8c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:18.065726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda2884-5e48-4aae-a29b-14edbc75f8c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:18.065989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda2884-5e48-4aae-a29b-14edbc75f8c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:18.075800 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda2884-5e48-4aae-a29b-14edbc75f8c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:18.077253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda2884-5e48-4aae-a29b-14edbc75f8c7 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:18.087082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1253/2505] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:18.507646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269093c2-5d7e-446f-89ed-0e2de648caeb None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:18.509122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-269093c2-5d7e-446f-89ed-0e2de648caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:18.522075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269093c2-5d7e-446f-89ed-0e2de648caeb None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:18.527811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:18.528722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:18.529005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:18.529279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] PATH_INFO: `/v3/projects/fd208bc38bbb4dea88bf582fb1c11e57` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:18.530788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd208bc38bbb4dea88bf582fb1c11e57)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:18.532962 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:18.587406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd208bc38bbb4dea88bf582fb1c11e57', 'request_id': 'req-a7a7c249-c843-4e17-98a6-22941918ac6e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:18.593735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd208bc38bbb4dea88bf582fb1c11e57', 'request_id': 'req-a7a7c249-c843-4e17-98a6-22941918ac6e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:18.602146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd208bc38bbb4dea88bf582fb1c11e57', 'request_id': 'req-a7a7c249-c843-4e17-98a6-22941918ac6e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:18.607011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:18.615236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] The token cache is being invalidate because project fd208bc38bbb4dea88bf582fb1c11e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:09:18.615703 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:18.616768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:18.618318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1253/2506] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:09:18 2022] DELETE /identity/v3/projects/fd208bc38bbb4dea88bf582fb1c11e57 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:18.622352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda2884-5e48-4aae-a29b-14edbc75f8c7 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:18.632376 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda2884-5e48-4aae-a29b-14edbc75f8c7 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:18.636755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:18.696078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:18.697031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:18.697319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:18.697603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] PATH_INFO: `/v3/users/3148fe9e3f894182a46c3d1d77c72476` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:18.698844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3148fe9e3f894182a46c3d1d77c72476)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:18.700842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:18.750133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3148fe9e3f894182a46c3d1d77c72476'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:18.751226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3148fe9e3f894182a46c3d1d77c72476'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:18.751851 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3148fe9e3f894182a46c3d1d77c72476'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:18.760135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3148fe9e3f894182a46c3d1d77c72476'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:18.772982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3148fe9e3f894182a46c3d1d77c72476'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:18.787490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1254/2507] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:09:18 2022] DELETE /identity/v3/users/3148fe9e3f894182a46c3d1d77c72476 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:19.129638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:19.131100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:19.140839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:19.143279 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:19.145449 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7c249-c843-4e17-98a6-22941918ac6e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:19.228689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:19.232055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.232584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.234333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] PATH_INFO: `/v3/users/f1bf26c2b8f64d4785bc51dc149adfa6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.235779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31ba30-5da7-4550-8ec5-5b50e7fd7468 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:19.235927 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1bf26c2b8f64d4785bc51dc149adfa6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:19.238460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:19.241333 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:19.242330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.242679 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.243163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] PATH_INFO: `/v3/projects/2051612041f9440fa1ed21a163a0b5d4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.244759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2051612041f9440fa1ed21a163a0b5d4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:19.246768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:19.285717 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:19.302663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1bf26c2b8f64d4785bc51dc149adfa6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:19.305733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2051612041f9440fa1ed21a163a0b5d4', 'request_id': 'req-16cd4aca-884b-49ee-892a-01503dd3df95'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:19.313245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1bf26c2b8f64d4785bc51dc149adfa6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:19.314212 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1bf26c2b8f64d4785bc51dc149adfa6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:19.317349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2051612041f9440fa1ed21a163a0b5d4', 'request_id': 'req-16cd4aca-884b-49ee-892a-01503dd3df95'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:19.323848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2051612041f9440fa1ed21a163a0b5d4', 'request_id': 'req-16cd4aca-884b-49ee-892a-01503dd3df95'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:19.329868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1bf26c2b8f64d4785bc51dc149adfa6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:19.335834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] The token cache is being invalidate because project 2051612041f9440fa1ed21a163a0b5d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:09:19.336154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:19.336957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-16cd4aca-884b-49ee-892a-01503dd3df95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:19.339329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1255/2508] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:09:19 2022] DELETE /identity/v3/projects/2051612041f9440fa1ed21a163a0b5d4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:09:19.343888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f389336a-b288-499d-b84d-770db05f76f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1bf26c2b8f64d4785bc51dc149adfa6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:19.344442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94deae2-8018-4fc5-b813-80a3d7ef6fe2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.344930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94deae2-8018-4fc5-b813-80a3d7ef6fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.345467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94deae2-8018-4fc5-b813-80a3d7ef6fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.352695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1254/2509] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:09:19 2022] DELETE /identity/v3/users/f1bf26c2b8f64d4785bc51dc149adfa6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:19.357107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c0f73-d813-485d-91b1-933d1ec3d006 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.357437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c0f73-d813-485d-91b1-933d1ec3d006 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.357796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c0f73-d813-485d-91b1-933d1ec3d006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.368215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c94deae2-8018-4fc5-b813-80a3d7ef6fe2 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:19.369722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c94deae2-8018-4fc5-b813-80a3d7ef6fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:19.376732 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-562c0f73-d813-485d-91b1-933d1ec3d006 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:19.378806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-562c0f73-d813-485d-91b1-933d1ec3d006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:19.471464 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1256/2510] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:09:19.475356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94deae2-8018-4fc5-b813-80a3d7ef6fe2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:19.479255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c94deae2-8018-4fc5-b813-80a3d7ef6fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:19.495080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1255/2511] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:19.498245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899be1b-d0dd-4d66-a628-44ce3941707f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.498245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899be1b-d0dd-4d66-a628-44ce3941707f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.498245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899be1b-d0dd-4d66-a628-44ce3941707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.521062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9899be1b-d0dd-4d66-a628-44ce3941707f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:19.522371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9899be1b-d0dd-4d66-a628-44ce3941707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:19.531397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1256/2512] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:19.535309 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899be1b-d0dd-4d66-a628-44ce3941707f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:19.536731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9899be1b-d0dd-4d66-a628-44ce3941707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:19.546158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:19.566076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94deae2-8018-4fc5-b813-80a3d7ef6fe2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:19.574578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd243e-94f9-41b8-bcff-ccea1b363bb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:19.576476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd243e-94f9-41b8-bcff-ccea1b363bb5 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.577095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd243e-94f9-41b8-bcff-ccea1b363bb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.577598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd243e-94f9-41b8-bcff-ccea1b363bb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.581550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-06bd243e-94f9-41b8-bcff-ccea1b363bb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:19.595977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd243e-94f9-41b8-bcff-ccea1b363bb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:19.597772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd243e-94f9-41b8-bcff-ccea1b363bb5 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:19.605451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899be1b-d0dd-4d66-a628-44ce3941707f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:19.609483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9242e73-c2ad-445b-a8fe-1db4a21ed88d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:19.609645 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1257/2513] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:09:19.609931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9242e73-c2ad-445b-a8fe-1db4a21ed88d service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.610215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9242e73-c2ad-445b-a8fe-1db4a21ed88d service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.610348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd243e-94f9-41b8-bcff-ccea1b363bb5 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:19.610628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9242e73-c2ad-445b-a8fe-1db4a21ed88d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.621483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd243e-94f9-41b8-bcff-ccea1b363bb5 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:19.621694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9242e73-c2ad-445b-a8fe-1db4a21ed88d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:19.623429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9242e73-c2ad-445b-a8fe-1db4a21ed88d service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:19.626879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a73c-e75b-4a40-a14c-516b22415d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:19.627547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a73c-e75b-4a40-a14c-516b22415d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.628405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a73c-e75b-4a40-a14c-516b22415d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.628805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a73c-e75b-4a40-a14c-516b22415d29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.629550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a73c-e75b-4a40-a14c-516b22415d29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:09:19.629863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a73c-e75b-4a40-a14c-516b22415d29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:19.631601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a73c-e75b-4a40-a14c-516b22415d29 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:19.636148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1257/2514] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:19.645550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1258/2515] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:09:19.653528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9242e73-c2ad-445b-a8fe-1db4a21ed88d service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:19.666272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9242e73-c2ad-445b-a8fe-1db4a21ed88d service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:19.673801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f5321-c65a-4315-9ba5-75f6b0dca17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:19.674503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f5321-c65a-4315-9ba5-75f6b0dca17b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.674988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f5321-c65a-4315-9ba5-75f6b0dca17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.675910 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f5321-c65a-4315-9ba5-75f6b0dca17b admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.677237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f5321-c65a-4315-9ba5-75f6b0dca17b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:19.679291 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f5321-c65a-4315-9ba5-75f6b0dca17b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:19.712126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1258/2516] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:19 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:19.716245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a73c-e75b-4a40-a14c-516b22415d29 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:19.728704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a73c-e75b-4a40-a14c-516b22415d29 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:19.757288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:19.796841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6b8bf-382f-4ea8-a64b-fe54a12da7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:19.797342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6b8bf-382f-4ea8-a64b-fe54a12da7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.797972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6b8bf-382f-4ea8-a64b-fe54a12da7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.797972 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6b8bf-382f-4ea8-a64b-fe54a12da7a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.798613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6b8bf-382f-4ea8-a64b-fe54a12da7a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:19.802262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6b8bf-382f-4ea8-a64b-fe54a12da7a6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:19.824229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1259/2517] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:19 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:19.828632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f5321-c65a-4315-9ba5-75f6b0dca17b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:19.840413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f5321-c65a-4315-9ba5-75f6b0dca17b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:19.847406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088edf1f-fcaa-4272-835e-f65a3f83be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:19.848461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088edf1f-fcaa-4272-835e-f65a3f83be7f admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.848751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088edf1f-fcaa-4272-835e-f65a3f83be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.849034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088edf1f-fcaa-4272-835e-f65a3f83be7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.849993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088edf1f-fcaa-4272-835e-f65a3f83be7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:19.851627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088edf1f-fcaa-4272-835e-f65a3f83be7f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:19.866147 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1259/2518] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 18:09:19.870421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6b8bf-382f-4ea8-a64b-fe54a12da7a6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:19.880770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6b8bf-382f-4ea8-a64b-fe54a12da7a6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:19.887332 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dbb1a-6d5b-499a-b524-83a9dcfb5f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:19.888006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dbb1a-6d5b-499a-b524-83a9dcfb5f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.888354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dbb1a-6d5b-499a-b524-83a9dcfb5f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.888634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dbb1a-6d5b-499a-b524-83a9dcfb5f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.889539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dbb1a-6d5b-499a-b524-83a9dcfb5f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:19.891137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dbb1a-6d5b-499a-b524-83a9dcfb5f99 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:19.903513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1260/2519] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 18:09:19.907460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088edf1f-fcaa-4272-835e-f65a3f83be7f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:19.921082 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088edf1f-fcaa-4272-835e-f65a3f83be7f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:19.926567 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b015e9-08c5-4d8e-b7c8-d4f6313be0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:19.927797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b015e9-08c5-4d8e-b7c8-d4f6313be0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.927944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b015e9-08c5-4d8e-b7c8-d4f6313be0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.928361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b015e9-08c5-4d8e-b7c8-d4f6313be0b2 admin admin] PATH_INFO: `/v3/projects/64e659a8bfef451fb1de70624606be12/users/2869f39f6e1b4fc1a41ab5f603132e54/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:19.951028 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b015e9-08c5-4d8e-b7c8-d4f6313be0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e659a8bfef451fb1de70624606be12, user_id=2869f39f6e1b4fc1a41ab5f603132e54, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:19.953950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b015e9-08c5-4d8e-b7c8-d4f6313be0b2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:19.969872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1260/2520] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:19 2022] PUT /identity/v3/projects/64e659a8bfef451fb1de70624606be12/users/2869f39f6e1b4fc1a41ab5f603132e54/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:19.992537 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89fb64-7eae-40f4-a90b-0fff31d69957 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:19.997208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89fb64-7eae-40f4-a90b-0fff31d69957 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:19.998586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89fb64-7eae-40f4-a90b-0fff31d69957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:20.035850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ed89fb64-7eae-40f4-a90b-0fff31d69957 None None] MFA Rules not processed for user `2869f39f6e1b4fc1a41ab5f603132e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:20.037393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ed89fb64-7eae-40f4-a90b-0fff31d69957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:20.134033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b015e9-08c5-4d8e-b7c8-d4f6313be0b2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:20.137141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-90b015e9-08c5-4d8e-b7c8-d4f6313be0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:20.153606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1261/2521] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:20.160310 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89fb64-7eae-40f4-a90b-0fff31d69957 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:20.256515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b015e9-08c5-4d8e-b7c8-d4f6313be0b2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:20.258436 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89fb64-7eae-40f4-a90b-0fff31d69957 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:20.261675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:20.262560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:20.262848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:20.263126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] PATH_INFO: `/v3/projects/8afc8e4291e6427dbae17b80c6b41f76` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:20.264622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8afc8e4291e6427dbae17b80c6b41f76)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:20.266547 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39469a08-b89a-476c-be16-7b734a1adaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:20.266703 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:20.267348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39469a08-b89a-476c-be16-7b734a1adaa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:20.267705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39469a08-b89a-476c-be16-7b734a1adaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:20.268053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39469a08-b89a-476c-be16-7b734a1adaa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:20.269463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39469a08-b89a-476c-be16-7b734a1adaa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:20.271535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39469a08-b89a-476c-be16-7b734a1adaa8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:20.300802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1262/2522] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:20 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:20.305216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39469a08-b89a-476c-be16-7b734a1adaa8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:20.330904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8afc8e4291e6427dbae17b80c6b41f76', 'request_id': 'req-79ffa819-12a0-4bfa-8260-635648ac0f92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:20.337844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:20.338929 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8afc8e4291e6427dbae17b80c6b41f76', 'request_id': 'req-79ffa819-12a0-4bfa-8260-635648ac0f92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:20.341281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:20.354724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8afc8e4291e6427dbae17b80c6b41f76', 'request_id': 'req-79ffa819-12a0-4bfa-8260-635648ac0f92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:20.371644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] The token cache is being invalidate because project 8afc8e4291e6427dbae17b80c6b41f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:09:20.371937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:20.372668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:20.375421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1261/2523] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:09:20 2022] DELETE /identity/v3/projects/8afc8e4291e6427dbae17b80c6b41f76 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:20.380223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:20.384564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:20.389324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39469a08-b89a-476c-be16-7b734a1adaa8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:20.464518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6dd67-a2d1-4f88-9dfa-0ad905d5b37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:20.465253 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6dd67-a2d1-4f88-9dfa-0ad905d5b37f admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:20.465769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6dd67-a2d1-4f88-9dfa-0ad905d5b37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:20.466366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6dd67-a2d1-4f88-9dfa-0ad905d5b37f admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:20.466707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6dd67-a2d1-4f88-9dfa-0ad905d5b37f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:20.469466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6dd67-a2d1-4f88-9dfa-0ad905d5b37f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:20.477760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffa819-12a0-4bfa-8260-635648ac0f92 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:20.485288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:20.486394 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:20.486778 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:20.487838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] PATH_INFO: `/v3/users/bd5d08a3a3e74ca79a5810230e1a6f17` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:20.489970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd5d08a3a3e74ca79a5810230e1a6f17)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:20.492416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:20.501979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1263/2524] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:20 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:20.507118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6dd67-a2d1-4f88-9dfa-0ad905d5b37f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:20.509923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-82f6dd67-a2d1-4f88-9dfa-0ad905d5b37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:20.529971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6dd67-a2d1-4f88-9dfa-0ad905d5b37f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:20.533158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39daf8-6a87-4dc6-95d5-2490a9dca6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:20.534526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39daf8-6a87-4dc6-95d5-2490a9dca6cc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:20.535218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39daf8-6a87-4dc6-95d5-2490a9dca6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:20.535218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39daf8-6a87-4dc6-95d5-2490a9dca6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:20.536069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39daf8-6a87-4dc6-95d5-2490a9dca6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:20.537900 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39daf8-6a87-4dc6-95d5-2490a9dca6cc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:20.555568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1264/2525] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:20.556623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd5d08a3a3e74ca79a5810230e1a6f17'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:20.557190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39daf8-6a87-4dc6-95d5-2490a9dca6cc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:20.564507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd5d08a3a3e74ca79a5810230e1a6f17'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:20.565696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd5d08a3a3e74ca79a5810230e1a6f17'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:20.569241 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39daf8-6a87-4dc6-95d5-2490a9dca6cc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:20.574255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efaaca3-1a31-4486-9db8-b139d38e3e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:20.575167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efaaca3-1a31-4486-9db8-b139d38e3e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:20.575451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efaaca3-1a31-4486-9db8-b139d38e3e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:20.575731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efaaca3-1a31-4486-9db8-b139d38e3e48 admin admin] PATH_INFO: `/v3/projects/65bea520bcfb44b4900ec184bf0165c7/users/4d336f1805084bc5926634b2e7fa3d2d/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:20.581426 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd5d08a3a3e74ca79a5810230e1a6f17'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:20.599023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd5d08a3a3e74ca79a5810230e1a6f17'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:20.602743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efaaca3-1a31-4486-9db8-b139d38e3e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65bea520bcfb44b4900ec184bf0165c7, user_id=4d336f1805084bc5926634b2e7fa3d2d, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:20.605513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efaaca3-1a31-4486-9db8-b139d38e3e48 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:20.606401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1262/2526] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:09:20 2022] DELETE /identity/v3/users/bd5d08a3a3e74ca79a5810230e1a6f17 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:09:20.622837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1265/2527] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:20 2022] PUT /identity/v3/projects/65bea520bcfb44b4900ec184bf0165c7/users/4d336f1805084bc5926634b2e7fa3d2d/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:20.626907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:20.628921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:20.734761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b07cf6-d20a-43b0-b08d-2f14e0ba27c3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:20.740898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:20.807236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fdb63-806b-4de9-82d8-39af277ca086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:20.807701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fdb63-806b-4de9-82d8-39af277ca086 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:20.807994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fdb63-806b-4de9-82d8-39af277ca086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:20.808259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fdb63-806b-4de9-82d8-39af277ca086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:20.809024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fdb63-806b-4de9-82d8-39af277ca086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:20.810776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fdb63-806b-4de9-82d8-39af277ca086 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:20.821576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1263/2528] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:20.825673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efaaca3-1a31-4486-9db8-b139d38e3e48 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:20.836631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efaaca3-1a31-4486-9db8-b139d38e3e48 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:20.841669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84fc45-77ed-41a2-b853-8700c7546980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:20.842411 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84fc45-77ed-41a2-b853-8700c7546980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:20.842823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84fc45-77ed-41a2-b853-8700c7546980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:20.843107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84fc45-77ed-41a2-b853-8700c7546980 admin admin] PATH_INFO: `/v3/projects/65bea520bcfb44b4900ec184bf0165c7/users/4d336f1805084bc5926634b2e7fa3d2d/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:20.845258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84fc45-77ed-41a2-b853-8700c7546980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65bea520bcfb44b4900ec184bf0165c7, user_id=4d336f1805084bc5926634b2e7fa3d2d, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:20.848779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84fc45-77ed-41a2-b853-8700c7546980 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:20.859511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1266/2529] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:20 2022] PUT /identity/v3/projects/65bea520bcfb44b4900ec184bf0165c7/users/4d336f1805084bc5926634b2e7fa3d2d/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:20.870950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32527f4-8b8b-487e-854b-9e03dd0398ec None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:20.871361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32527f4-8b8b-487e-854b-9e03dd0398ec None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:20.871792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32527f4-8b8b-487e-854b-9e03dd0398ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:20.901673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f32527f4-8b8b-487e-854b-9e03dd0398ec None None] MFA Rules not processed for user `4d336f1805084bc5926634b2e7fa3d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:20.903025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f32527f4-8b8b-487e-854b-9e03dd0398ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:20.930038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84fc45-77ed-41a2-b853-8700c7546980 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:20.933922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6a84fc45-77ed-41a2-b853-8700c7546980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:20.984926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1264/2530] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:20.996979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32527f4-8b8b-487e-854b-9e03dd0398ec None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:20.998329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f32527f4-8b8b-487e-854b-9e03dd0398ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:21.012580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84fc45-77ed-41a2-b853-8700c7546980 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.017196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.018064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.018373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.018650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] PATH_INFO: `/v3/projects/50bd1ff063fa43e0ba4a9e1bf5f13edf` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.019848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50bd1ff063fa43e0ba4a9e1bf5f13edf)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.021934 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.033399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:21.061383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32527f4-8b8b-487e-854b-9e03dd0398ec None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.065943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d24c00-2d88-4dfb-92f2-f02550b8eaa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.066761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d24c00-2d88-4dfb-92f2-f02550b8eaa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.067282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d24c00-2d88-4dfb-92f2-f02550b8eaa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.067559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d24c00-2d88-4dfb-92f2-f02550b8eaa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.077061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d24c00-2d88-4dfb-92f2-f02550b8eaa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.078682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d24c00-2d88-4dfb-92f2-f02550b8eaa9 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.079888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50bd1ff063fa43e0ba4a9e1bf5f13edf', 'request_id': 'req-9b96dd95-202f-4c91-bde3-9b438bd1d036'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:21.088633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50bd1ff063fa43e0ba4a9e1bf5f13edf', 'request_id': 'req-9b96dd95-202f-4c91-bde3-9b438bd1d036'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:21.090242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1265/2531] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:20 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:09:21.095042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50bd1ff063fa43e0ba4a9e1bf5f13edf', 'request_id': 'req-9b96dd95-202f-4c91-bde3-9b438bd1d036'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:21.106326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] The token cache is being invalidate because project 50bd1ff063fa43e0ba4a9e1bf5f13edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:09:21.106614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:21.107302 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:21.109197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1267/2532] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:09:20 2022] DELETE /identity/v3/projects/50bd1ff063fa43e0ba4a9e1bf5f13edf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:09:21.114550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a01d3-7123-4e40-8468-2cc919e7bd7b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.114834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a01d3-7123-4e40-8468-2cc919e7bd7b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.115114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a01d3-7123-4e40-8468-2cc919e7bd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.139389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e53a01d3-7123-4e40-8468-2cc919e7bd7b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:21.140639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e53a01d3-7123-4e40-8468-2cc919e7bd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:21.163877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.165646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:21.173491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:21.204817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1266/2533] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:21.214716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c53d9a-ec59-4804-bf86-eb74199b196d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.215029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c53d9a-ec59-4804-bf86-eb74199b196d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.215307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c53d9a-ec59-4804-bf86-eb74199b196d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.229380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96dd95-202f-4c91-bde3-9b438bd1d036 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.233004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fd152-6003-4d2c-a14e-c731fd357ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.233155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-22c53d9a-ec59-4804-bf86-eb74199b196d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:21.233655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fd152-6003-4d2c-a14e-c731fd357ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.233941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fd152-6003-4d2c-a14e-c731fd357ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.234221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-22c53d9a-ec59-4804-bf86-eb74199b196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:21.234339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fd152-6003-4d2c-a14e-c731fd357ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.236091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-571fd152-6003-4d2c-a14e-c731fd357ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:21.242352 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1267/2534] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:09:21.246458 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c53d9a-ec59-4804-bf86-eb74199b196d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.256033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c53d9a-ec59-4804-bf86-eb74199b196d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.260851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c7467-803a-438e-85de-bb37701b8648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.261491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c7467-803a-438e-85de-bb37701b8648 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.261772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c7467-803a-438e-85de-bb37701b8648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.262046 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c7467-803a-438e-85de-bb37701b8648 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.262761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c7467-803a-438e-85de-bb37701b8648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:09:21.263068 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c7467-803a-438e-85de-bb37701b8648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.264728 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c7467-803a-438e-85de-bb37701b8648 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.277085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1268/2535] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 18:09:21.281053 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c7467-803a-438e-85de-bb37701b8648 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.290541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c7467-803a-438e-85de-bb37701b8648 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.295002 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce90471-bd34-4780-9862-8108b0daec71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.295606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce90471-bd34-4780-9862-8108b0daec71 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.295886 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce90471-bd34-4780-9862-8108b0daec71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.296164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce90471-bd34-4780-9862-8108b0daec71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.296877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce90471-bd34-4780-9862-8108b0daec71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.298358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fd152-6003-4d2c-a14e-c731fd357ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.298855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce90471-bd34-4780-9862-8108b0daec71 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.299799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fd152-6003-4d2c-a14e-c731fd357ee9 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.308813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1268/2536] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:09:21.326125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1269/2537] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:21 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:09:21.330345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fd152-6003-4d2c-a14e-c731fd357ee9 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.341966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fd152-6003-4d2c-a14e-c731fd357ee9 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.400726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:21.409326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eff6f-1981-4d3d-ab82-6c43cbc5bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.410113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eff6f-1981-4d3d-ab82-6c43cbc5bb97 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.410392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eff6f-1981-4d3d-ab82-6c43cbc5bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.410912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eff6f-1981-4d3d-ab82-6c43cbc5bb97 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.412200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eff6f-1981-4d3d-ab82-6c43cbc5bb97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.415300 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eff6f-1981-4d3d-ab82-6c43cbc5bb97 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.442079 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1269/2538] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:21 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:21.445787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce90471-bd34-4780-9862-8108b0daec71 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.460665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce90471-bd34-4780-9862-8108b0daec71 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.465729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd511f94-d3e3-45ef-a2d5-561b32f61e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.466418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511f94-d3e3-45ef-a2d5-561b32f61e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.466709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511f94-d3e3-45ef-a2d5-561b32f61e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.466984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511f94-d3e3-45ef-a2d5-561b32f61e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.467714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd511f94-d3e3-45ef-a2d5-561b32f61e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.469410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd511f94-d3e3-45ef-a2d5-561b32f61e18 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.480244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1270/2539] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 18:09:21.484609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eff6f-1981-4d3d-ab82-6c43cbc5bb97 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.495963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eff6f-1981-4d3d-ab82-6c43cbc5bb97 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.500572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f40772-70fd-468a-9996-8cdd070f9834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.501235 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f40772-70fd-468a-9996-8cdd070f9834 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.501521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f40772-70fd-468a-9996-8cdd070f9834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.501810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f40772-70fd-468a-9996-8cdd070f9834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.502541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f40772-70fd-468a-9996-8cdd070f9834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.504838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f40772-70fd-468a-9996-8cdd070f9834 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.514790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1270/2540] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:21.519425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd511f94-d3e3-45ef-a2d5-561b32f61e18 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.529253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd511f94-d3e3-45ef-a2d5-561b32f61e18 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.533981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071ab41-046d-496c-a56b-e5e20685f68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.535494 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071ab41-046d-496c-a56b-e5e20685f68f admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.535786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071ab41-046d-496c-a56b-e5e20685f68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.536065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071ab41-046d-496c-a56b-e5e20685f68f admin admin] PATH_INFO: `/v3/projects/2ebbb3f0fd6a4a708b08fedc8ab8100c/users/7153905a0ea949118ab83e870c6fa659/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.553479 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071ab41-046d-496c-a56b-e5e20685f68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ebbb3f0fd6a4a708b08fedc8ab8100c, user_id=7153905a0ea949118ab83e870c6fa659, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.556157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071ab41-046d-496c-a56b-e5e20685f68f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.570317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1271/2541] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:21 2022] PUT /identity/v3/projects/2ebbb3f0fd6a4a708b08fedc8ab8100c/users/7153905a0ea949118ab83e870c6fa659/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:21.580649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd248c80-0cf9-4aa6-8b7e-b17b65e5347a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.581008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd248c80-0cf9-4aa6-8b7e-b17b65e5347a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.581690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd248c80-0cf9-4aa6-8b7e-b17b65e5347a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.607507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-bd248c80-0cf9-4aa6-8b7e-b17b65e5347a None None] MFA Rules not processed for user `7153905a0ea949118ab83e870c6fa659`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:21.608742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bd248c80-0cf9-4aa6-8b7e-b17b65e5347a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:21.673114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1271/2542] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:21.678769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071ab41-046d-496c-a56b-e5e20685f68f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.739730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071ab41-046d-496c-a56b-e5e20685f68f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.744280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba58b03-ce90-4f35-acea-5401f9dd0525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.744955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58b03-ce90-4f35-acea-5401f9dd0525 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.745237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58b03-ce90-4f35-acea-5401f9dd0525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.745514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58b03-ce90-4f35-acea-5401f9dd0525 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.746247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba58b03-ce90-4f35-acea-5401f9dd0525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.748089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba58b03-ce90-4f35-acea-5401f9dd0525 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.772836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1272/2543] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:21 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:21.776821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd248c80-0cf9-4aa6-8b7e-b17b65e5347a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.787961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd248c80-0cf9-4aa6-8b7e-b17b65e5347a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.842631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc13a1-6b74-4381-a226-6bab908f51ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.843302 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc13a1-6b74-4381-a226-6bab908f51ce admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.843578 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc13a1-6b74-4381-a226-6bab908f51ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.843859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc13a1-6b74-4381-a226-6bab908f51ce admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.844594 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc13a1-6b74-4381-a226-6bab908f51ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.847023 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc13a1-6b74-4381-a226-6bab908f51ce admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.869195 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1272/2544] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:21 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:21.873275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba58b03-ce90-4f35-acea-5401f9dd0525 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.883483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba58b03-ce90-4f35-acea-5401f9dd0525 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.888012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0a403-77ea-401b-8fae-f8faba0de060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.888700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0a403-77ea-401b-8fae-f8faba0de060 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.888984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0a403-77ea-401b-8fae-f8faba0de060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.889260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0a403-77ea-401b-8fae-f8faba0de060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.889983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0a403-77ea-401b-8fae-f8faba0de060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.891587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0a403-77ea-401b-8fae-f8faba0de060 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.902001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1273/2545] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:21.905884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc13a1-6b74-4381-a226-6bab908f51ce admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.916377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc13a1-6b74-4381-a226-6bab908f51ce admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.921074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad81f0-1bcb-4121-980c-029980b488b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:21.921961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad81f0-1bcb-4121-980c-029980b488b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:21.922246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad81f0-1bcb-4121-980c-029980b488b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:21.922529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad81f0-1bcb-4121-980c-029980b488b3 admin admin] PATH_INFO: `/v3/projects/cc03a25f283d454c8ded49265b474d98/users/296f5ccf6da74493a23ef0bf0e047d55/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:21.938283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad81f0-1bcb-4121-980c-029980b488b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc03a25f283d454c8ded49265b474d98, user_id=296f5ccf6da74493a23ef0bf0e047d55, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:21.941270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad81f0-1bcb-4121-980c-029980b488b3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:21.953856 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1273/2546] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:21 2022] PUT /identity/v3/projects/cc03a25f283d454c8ded49265b474d98/users/296f5ccf6da74493a23ef0bf0e047d55/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:21.957773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0a403-77ea-401b-8fae-f8faba0de060 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:21.967690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0a403-77ea-401b-8fae-f8faba0de060 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:21.996619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:22.024722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a84e1f-247b-4138-a68d-d1572a33ecfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:22.025380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a84e1f-247b-4138-a68d-d1572a33ecfa admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:22.025658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a84e1f-247b-4138-a68d-d1572a33ecfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:22.025931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a84e1f-247b-4138-a68d-d1572a33ecfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:22.026652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a84e1f-247b-4138-a68d-d1572a33ecfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:22.028251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a84e1f-247b-4138-a68d-d1572a33ecfa admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:22.038457 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1274/2547] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:22.042514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad81f0-1bcb-4121-980c-029980b488b3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:22.053599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad81f0-1bcb-4121-980c-029980b488b3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:22.058174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d3e74-21c7-4020-8c73-b7aeca332cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:22.059047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d3e74-21c7-4020-8c73-b7aeca332cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:22.059327 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d3e74-21c7-4020-8c73-b7aeca332cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:22.059602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d3e74-21c7-4020-8c73-b7aeca332cf7 admin admin] PATH_INFO: `/v3/projects/cc03a25f283d454c8ded49265b474d98/users/296f5ccf6da74493a23ef0bf0e047d55/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:22.061583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d3e74-21c7-4020-8c73-b7aeca332cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc03a25f283d454c8ded49265b474d98, user_id=296f5ccf6da74493a23ef0bf0e047d55, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:22.064214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d3e74-21c7-4020-8c73-b7aeca332cf7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:22.074440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1274/2548] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:22 2022] PUT /identity/v3/projects/cc03a25f283d454c8ded49265b474d98/users/296f5ccf6da74493a23ef0bf0e047d55/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:22.084514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d833b7-1b9a-401b-b43f-35d3e8a2325e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:22.084812 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d833b7-1b9a-401b-b43f-35d3e8a2325e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:22.085100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d833b7-1b9a-401b-b43f-35d3e8a2325e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:22.113242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-25d833b7-1b9a-401b-b43f-35d3e8a2325e None None] MFA Rules not processed for user `296f5ccf6da74493a23ef0bf0e047d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:22.114455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-25d833b7-1b9a-401b-b43f-35d3e8a2325e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:22.181581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1275/2549] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:22.192927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d3e74-21c7-4020-8c73-b7aeca332cf7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:22.194633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c70d3e74-21c7-4020-8c73-b7aeca332cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:22.213312 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:22.251305 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d3e74-21c7-4020-8c73-b7aeca332cf7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:22.255502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e6a28-8b62-4701-8778-8eaed77fd2a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:22.256134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e6a28-8b62-4701-8778-8eaed77fd2a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:22.256450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e6a28-8b62-4701-8778-8eaed77fd2a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:22.256727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e6a28-8b62-4701-8778-8eaed77fd2a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:22.312482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e6a28-8b62-4701-8778-8eaed77fd2a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:22.313913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e6a28-8b62-4701-8778-8eaed77fd2a4 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:22.323617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1275/2550] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:27.948349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d833b7-1b9a-401b-b43f-35d3e8a2325e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:27.958657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d833b7-1b9a-401b-b43f-35d3e8a2325e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:27.962152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba9f94-66fa-4c9c-8271-e1927a81a6d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:27.962780 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba9f94-66fa-4c9c-8271-e1927a81a6d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:27.963059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba9f94-66fa-4c9c-8271-e1927a81a6d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:27.963335 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba9f94-66fa-4c9c-8271-e1927a81a6d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:27.972626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba9f94-66fa-4c9c-8271-e1927a81a6d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:27.974092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba9f94-66fa-4c9c-8271-e1927a81a6d5 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:27.983503 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1276/2551] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:32.319739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=41 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:37.979283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=55 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:56.918514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e6a28-8b62-4701-8778-8eaed77fd2a4 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:56.980838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e6a28-8b62-4701-8778-8eaed77fd2a4 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:56.985484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:56.986357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:56.986644 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:56.986924 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] PATH_INFO: `/v3/users/7153905a0ea949118ab83e870c6fa659` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:56.988398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7153905a0ea949118ab83e870c6fa659)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:56.990494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:57.037993 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7153905a0ea949118ab83e870c6fa659'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.038673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7153905a0ea949118ab83e870c6fa659'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.039462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7153905a0ea949118ab83e870c6fa659'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.046375 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7153905a0ea949118ab83e870c6fa659'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.058209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7153905a0ea949118ab83e870c6fa659'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.073936 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1276/2552] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:09:56 2022] DELETE /identity/v3/users/7153905a0ea949118ab83e870c6fa659 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:57.078406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebb4f4-f697-465a-9da4-6b8150b67f97 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:57.078873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebb4f4-f697-465a-9da4-6b8150b67f97 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:57.079093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebb4f4-f697-465a-9da4-6b8150b67f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:57.097791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-cbebb4f4-f697-465a-9da4-6b8150b67f97 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:57.098991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cbebb4f4-f697-465a-9da4-6b8150b67f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:57.159706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1277/2553] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:57.169995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:57.170749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:57.230521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64658799-8127-4122-8258-55ea1c74e56b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:57.234208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abd433-c856-47e2-a0f6-3186ca9a4f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:57.234863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abd433-c856-47e2-a0f6-3186ca9a4f7a service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:57.235146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abd433-c856-47e2-a0f6-3186ca9a4f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:57.235492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abd433-c856-47e2-a0f6-3186ca9a4f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:57.246145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04abd433-c856-47e2-a0f6-3186ca9a4f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:57.247640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04abd433-c856-47e2-a0f6-3186ca9a4f7a service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:57.259072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1277/2554] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:57.545416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebb4f4-f697-465a-9da4-6b8150b67f97 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:57.547016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cbebb4f4-f697-465a-9da4-6b8150b67f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:57.559777 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebb4f4-f697-465a-9da4-6b8150b67f97 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:57.565934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:57.568265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:57.568592 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:57.568866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] PATH_INFO: `/v3/projects/2ebbb3f0fd6a4a708b08fedc8ab8100c` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:57.570187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ebbb3f0fd6a4a708b08fedc8ab8100c)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:57.572931 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:57.623092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ebbb3f0fd6a4a708b08fedc8ab8100c', 'request_id': 'req-ef31e197-7600-4e09-8121-671c01c7cfe8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.630300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ebbb3f0fd6a4a708b08fedc8ab8100c', 'request_id': 'req-ef31e197-7600-4e09-8121-671c01c7cfe8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.638561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ebbb3f0fd6a4a708b08fedc8ab8100c', 'request_id': 'req-ef31e197-7600-4e09-8121-671c01c7cfe8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.649151 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] The token cache is being invalidate because project 2ebbb3f0fd6a4a708b08fedc8ab8100c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:09:57.649438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.650065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.651647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1278/2555] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:09:57 2022] DELETE /identity/v3/projects/2ebbb3f0fd6a4a708b08fedc8ab8100c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:57.655620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abd433-c856-47e2-a0f6-3186ca9a4f7a service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:57.665730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abd433-c856-47e2-a0f6-3186ca9a4f7a service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:57.686632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:57.729727 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:57.730592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:57.730870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:57.731140 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] PATH_INFO: `/v3/users/296f5ccf6da74493a23ef0bf0e047d55` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:57.732344 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=296f5ccf6da74493a23ef0bf0e047d55)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:57.734286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:57.777968 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '296f5ccf6da74493a23ef0bf0e047d55'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.778604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '296f5ccf6da74493a23ef0bf0e047d55'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.779199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '296f5ccf6da74493a23ef0bf0e047d55'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.786463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '296f5ccf6da74493a23ef0bf0e047d55'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.797872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b19db49-bd32-44ab-a349-891e08066aa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '296f5ccf6da74493a23ef0bf0e047d55'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:57.811864 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1278/2556] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:09:57 2022] DELETE /identity/v3/users/296f5ccf6da74493a23ef0bf0e047d55 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:58.076838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:58.078286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:58.092026 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31e197-7600-4e09-8121-671c01c7cfe8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:58.148410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:58.149273 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.149557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.149832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] PATH_INFO: `/v3/projects/cc03a25f283d454c8ded49265b474d98` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.150996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc03a25f283d454c8ded49265b474d98)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:58.152954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:58.182872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:58.201696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc03a25f283d454c8ded49265b474d98', 'request_id': 'req-d65b2952-ec25-4694-83e6-08803eeaa525'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:58.207748 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc03a25f283d454c8ded49265b474d98', 'request_id': 'req-d65b2952-ec25-4694-83e6-08803eeaa525'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:58.214996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc03a25f283d454c8ded49265b474d98', 'request_id': 'req-d65b2952-ec25-4694-83e6-08803eeaa525'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:58.225315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] The token cache is being invalidate because project cc03a25f283d454c8ded49265b474d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:09:58.225605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:58.226231 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d65b2952-ec25-4694-83e6-08803eeaa525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:09:58.227751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1279/2557] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:09:58 2022] DELETE /identity/v3/projects/cc03a25f283d454c8ded49265b474d98 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:58.232587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af61ca-e728-49c6-91dc-2730abe3bdf6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.233070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af61ca-e728-49c6-91dc-2730abe3bdf6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.233177 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af61ca-e728-49c6-91dc-2730abe3bdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.250731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d8af61ca-e728-49c6-91dc-2730abe3bdf6 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:58.251915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d8af61ca-e728-49c6-91dc-2730abe3bdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:58.309860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1279/2558] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:58.318373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dac65-e19f-4d0d-904f-9b74d7aa6765 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.318685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dac65-e19f-4d0d-904f-9b74d7aa6765 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.318972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dac65-e19f-4d0d-904f-9b74d7aa6765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.337376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-437dac65-e19f-4d0d-904f-9b74d7aa6765 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:58.338578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-437dac65-e19f-4d0d-904f-9b74d7aa6765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:58.346424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1280/2559] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:58.350397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af61ca-e728-49c6-91dc-2730abe3bdf6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:58.359979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af61ca-e728-49c6-91dc-2730abe3bdf6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:58.365199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8a09f-e083-41d5-ba22-631b2155bad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:58.365854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a09f-e083-41d5-ba22-631b2155bad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.366138 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a09f-e083-41d5-ba22-631b2155bad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.366415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a09f-e083-41d5-ba22-631b2155bad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.367148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8a09f-e083-41d5-ba22-631b2155bad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:09:58.367456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8a09f-e083-41d5-ba22-631b2155bad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:58.369090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8a09f-e083-41d5-ba22-631b2155bad7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:58.381102 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1280/2560] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:09:58.385389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dac65-e19f-4d0d-904f-9b74d7aa6765 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:58.394492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dac65-e19f-4d0d-904f-9b74d7aa6765 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:58.399115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce74141-5909-4516-8b6f-b91d3d6f195c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:58.399822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce74141-5909-4516-8b6f-b91d3d6f195c admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.400148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce74141-5909-4516-8b6f-b91d3d6f195c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.400451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce74141-5909-4516-8b6f-b91d3d6f195c admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.401150 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce74141-5909-4516-8b6f-b91d3d6f195c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:58.403077 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce74141-5909-4516-8b6f-b91d3d6f195c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:58.428910 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1281/2561] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:58 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:58.432732 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8a09f-e083-41d5-ba22-631b2155bad7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:58.443324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8a09f-e083-41d5-ba22-631b2155bad7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:58.470462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:58.519906 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b7d4e-ee7e-4b04-b053-c274c151e639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:58.520619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b7d4e-ee7e-4b04-b053-c274c151e639 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.520901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b7d4e-ee7e-4b04-b053-c274c151e639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.521176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b7d4e-ee7e-4b04-b053-c274c151e639 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.521878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b7d4e-ee7e-4b04-b053-c274c151e639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:58.523833 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b7d4e-ee7e-4b04-b053-c274c151e639 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:58.544274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1281/2562] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:58 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:58.548390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce74141-5909-4516-8b6f-b91d3d6f195c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:58.559373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce74141-5909-4516-8b6f-b91d3d6f195c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:58.563833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c395fe5-1e22-449d-a18a-f38d5384b6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:58.564516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c395fe5-1e22-449d-a18a-f38d5384b6ac admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.564798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c395fe5-1e22-449d-a18a-f38d5384b6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.565152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c395fe5-1e22-449d-a18a-f38d5384b6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.566032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c395fe5-1e22-449d-a18a-f38d5384b6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:58.567721 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c395fe5-1e22-449d-a18a-f38d5384b6ac admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:58.578378 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1282/2563] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:58.582414 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b7d4e-ee7e-4b04-b053-c274c151e639 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:58.592546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b7d4e-ee7e-4b04-b053-c274c151e639 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:58.598181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c009af-b81d-45ea-87f8-e63b625a3a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:58.598860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c009af-b81d-45ea-87f8-e63b625a3a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.599148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c009af-b81d-45ea-87f8-e63b625a3a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.599431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c009af-b81d-45ea-87f8-e63b625a3a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.600171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c009af-b81d-45ea-87f8-e63b625a3a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:58.601824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c009af-b81d-45ea-87f8-e63b625a3a73 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:58.613060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1282/2564] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:58.617420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c395fe5-1e22-449d-a18a-f38d5384b6ac admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:58.632501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c395fe5-1e22-449d-a18a-f38d5384b6ac admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:58.632501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616f247-a98f-4135-b4ee-bea2ec160887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:58.633367 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616f247-a98f-4135-b4ee-bea2ec160887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.633648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616f247-a98f-4135-b4ee-bea2ec160887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.634075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616f247-a98f-4135-b4ee-bea2ec160887 admin admin] PATH_INFO: `/v3/projects/1f7bab4d926040e3a63aa5c35443977a/users/ed9ac6c572d34bb9bfb930cece53f337/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.649327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616f247-a98f-4135-b4ee-bea2ec160887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7bab4d926040e3a63aa5c35443977a, user_id=ed9ac6c572d34bb9bfb930cece53f337, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:58.652096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616f247-a98f-4135-b4ee-bea2ec160887 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:58.667482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1283/2565] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:58 2022] PUT /identity/v3/projects/1f7bab4d926040e3a63aa5c35443977a/users/ed9ac6c572d34bb9bfb930cece53f337/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:58.672835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363022a-e8aa-495e-81a2-568208f18a75 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.673174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363022a-e8aa-495e-81a2-568208f18a75 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.673455 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363022a-e8aa-495e-81a2-568208f18a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.692009 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f363022a-e8aa-495e-81a2-568208f18a75 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:58.693158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f363022a-e8aa-495e-81a2-568208f18a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:58.755802 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1283/2566] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:58.765522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616f247-a98f-4135-b4ee-bea2ec160887 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:58.767987 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6616f247-a98f-4135-b4ee-bea2ec160887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:58.826027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616f247-a98f-4135-b4ee-bea2ec160887 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:58.829671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab82c3-1f41-4c9c-b873-48d4f793b058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:58.830310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab82c3-1f41-4c9c-b873-48d4f793b058 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:58.831146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab82c3-1f41-4c9c-b873-48d4f793b058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:58.831373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab82c3-1f41-4c9c-b873-48d4f793b058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:58.840264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab82c3-1f41-4c9c-b873-48d4f793b058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:58.841727 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab82c3-1f41-4c9c-b873-48d4f793b058 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:58.851738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1284/2567] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:09:59.484487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ab8e2-dbf3-427c-9471-3a8472c2d5fc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:59.484991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ab8e2-dbf3-427c-9471-3a8472c2d5fc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:59.485108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ab8e2-dbf3-427c-9471-3a8472c2d5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:59.512498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a26ab8e2-dbf3-427c-9471-3a8472c2d5fc None None] MFA Rules not processed for user `ed9ac6c572d34bb9bfb930cece53f337`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:09:59.513642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a26ab8e2-dbf3-427c-9471-3a8472c2d5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:09:59.591217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1284/2568] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:09:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:09:59.595674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab82c3-1f41-4c9c-b873-48d4f793b058 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:59.605524 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab82c3-1f41-4c9c-b873-48d4f793b058 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:59.610049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72565cac-73b1-48e5-bcbd-9366b0194879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:59.610725 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72565cac-73b1-48e5-bcbd-9366b0194879 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:59.611007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72565cac-73b1-48e5-bcbd-9366b0194879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:59.611279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72565cac-73b1-48e5-bcbd-9366b0194879 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:59.612538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72565cac-73b1-48e5-bcbd-9366b0194879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:59.614429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72565cac-73b1-48e5-bcbd-9366b0194879 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:59.641817 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1285/2569] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:09:59 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:09:59.646248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ab8e2-dbf3-427c-9471-3a8472c2d5fc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:59.656872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ab8e2-dbf3-427c-9471-3a8472c2d5fc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:59.691875 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:59.715428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31e2f-5acd-4009-90a7-c14dfc2f45c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:59.716097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31e2f-5acd-4009-90a7-c14dfc2f45c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:59.716409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31e2f-5acd-4009-90a7-c14dfc2f45c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:59.716684 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31e2f-5acd-4009-90a7-c14dfc2f45c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:59.717401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c31e2f-5acd-4009-90a7-c14dfc2f45c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:59.719457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c31e2f-5acd-4009-90a7-c14dfc2f45c3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:59.739600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1285/2570] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:09:59 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:09:59.743607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72565cac-73b1-48e5-bcbd-9366b0194879 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:59.754754 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72565cac-73b1-48e5-bcbd-9366b0194879 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:59.759225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e37df-9298-41cd-adb3-7e6210c1544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:59.759872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e37df-9298-41cd-adb3-7e6210c1544e admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:59.760155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e37df-9298-41cd-adb3-7e6210c1544e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:59.760492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e37df-9298-41cd-adb3-7e6210c1544e admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:59.761215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e37df-9298-41cd-adb3-7e6210c1544e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:59.762853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e37df-9298-41cd-adb3-7e6210c1544e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:59.773306 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1286/2571] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:59.777250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31e2f-5acd-4009-90a7-c14dfc2f45c3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:59.786906 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31e2f-5acd-4009-90a7-c14dfc2f45c3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:59.791447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc4caf-134d-45ab-b8a9-71ac117fc0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:59.792341 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc4caf-134d-45ab-b8a9-71ac117fc0c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:59.792625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc4caf-134d-45ab-b8a9-71ac117fc0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:59.792899 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc4caf-134d-45ab-b8a9-71ac117fc0c9 admin admin] PATH_INFO: `/v3/projects/dc4077e272394552ad59e5624a63a351/users/cdbc56c254864795894b3298faf53ca0/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:59.808054 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc4caf-134d-45ab-b8a9-71ac117fc0c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc4077e272394552ad59e5624a63a351, user_id=cdbc56c254864795894b3298faf53ca0, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:59.811092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc4caf-134d-45ab-b8a9-71ac117fc0c9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:59.824957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1286/2572] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:59 2022] PUT /identity/v3/projects/dc4077e272394552ad59e5624a63a351/users/cdbc56c254864795894b3298faf53ca0/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:09:59.828866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e37df-9298-41cd-adb3-7e6210c1544e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:59.839638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e37df-9298-41cd-adb3-7e6210c1544e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:59.856642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:09:59.904213 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf1650-c436-4e07-9ed3-c66a18624ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:59.904981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf1650-c436-4e07-9ed3-c66a18624ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:59.905253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf1650-c436-4e07-9ed3-c66a18624ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:59.905521 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf1650-c436-4e07-9ed3-c66a18624ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:59.906243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf1650-c436-4e07-9ed3-c66a18624ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:59.908118 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf1650-c436-4e07-9ed3-c66a18624ff7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:59.925180 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1287/2573] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:09:59.930046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc4caf-134d-45ab-b8a9-71ac117fc0c9 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:09:59.941463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc4caf-134d-45ab-b8a9-71ac117fc0c9 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:09:59.946378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47cce4-7e00-47e1-a323-f20f005d65f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:09:59.947269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47cce4-7e00-47e1-a323-f20f005d65f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:09:59.947552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47cce4-7e00-47e1-a323-f20f005d65f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:09:59.947828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47cce4-7e00-47e1-a323-f20f005d65f6 admin admin] PATH_INFO: `/v3/projects/dc4077e272394552ad59e5624a63a351/users/cdbc56c254864795894b3298faf53ca0/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:09:59.949986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47cce4-7e00-47e1-a323-f20f005d65f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc4077e272394552ad59e5624a63a351, user_id=cdbc56c254864795894b3298faf53ca0, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:09:59.952634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47cce4-7e00-47e1-a323-f20f005d65f6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:09:59.964548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1287/2574] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:09:59 2022] PUT /identity/v3/projects/dc4077e272394552ad59e5624a63a351/users/cdbc56c254864795894b3298faf53ca0/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:00.606322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5bb38-3c57-4477-bd4e-f8ecd31d1680 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:00.607299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5bb38-3c57-4477-bd4e-f8ecd31d1680 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:00.607764 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5bb38-3c57-4477-bd4e-f8ecd31d1680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:00.635666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-94d5bb38-3c57-4477-bd4e-f8ecd31d1680 None None] MFA Rules not processed for user `cdbc56c254864795894b3298faf53ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:00.636978 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-94d5bb38-3c57-4477-bd4e-f8ecd31d1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:00.708734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1288/2575] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:00 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:00.720663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47cce4-7e00-47e1-a323-f20f005d65f6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:00.722431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7c47cce4-7e00-47e1-a323-f20f005d65f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:00.781228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47cce4-7e00-47e1-a323-f20f005d65f6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:00.784963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05eb6b-02cd-4db4-ba5c-bf13d3db730c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:00.785619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05eb6b-02cd-4db4-ba5c-bf13d3db730c service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:00.785903 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05eb6b-02cd-4db4-ba5c-bf13d3db730c service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:00.786183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05eb6b-02cd-4db4-ba5c-bf13d3db730c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:00.843608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05eb6b-02cd-4db4-ba5c-bf13d3db730c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:00.845075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05eb6b-02cd-4db4-ba5c-bf13d3db730c service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:00.854036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1288/2576] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:02.206140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5bb38-3c57-4477-bd4e-f8ecd31d1680 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:02.207577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-94d5bb38-3c57-4477-bd4e-f8ecd31d1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:02.220137 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5bb38-3c57-4477-bd4e-f8ecd31d1680 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:02.223617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b15cb6-4d6b-4b5a-829c-24783243a5a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:02.224482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b15cb6-4d6b-4b5a-829c-24783243a5a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:02.224762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b15cb6-4d6b-4b5a-829c-24783243a5a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:02.225061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b15cb6-4d6b-4b5a-829c-24783243a5a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:02.233829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b15cb6-4d6b-4b5a-829c-24783243a5a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:02.235247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b15cb6-4d6b-4b5a-829c-24783243a5a6 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:02.244680 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1289/2577] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:02.699254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05eb6b-02cd-4db4-ba5c-bf13d3db730c service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:02.709601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05eb6b-02cd-4db4-ba5c-bf13d3db730c service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:02.713808 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d24fe-6cbf-4c27-9c96-31606e7a686d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:02.714540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d24fe-6cbf-4c27-9c96-31606e7a686d service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:02.714821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d24fe-6cbf-4c27-9c96-31606e7a686d service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:02.715098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d24fe-6cbf-4c27-9c96-31606e7a686d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:02.716909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-763d24fe-6cbf-4c27-9c96-31606e7a686d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:02.765285 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:02.804971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d24fe-6cbf-4c27-9c96-31606e7a686d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:02.806497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d24fe-6cbf-4c27-9c96-31606e7a686d service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:02.826794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1289/2578] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:10:09.898262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b15cb6-4d6b-4b5a-829c-24783243a5a6 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:09.899773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-89b15cb6-4d6b-4b5a-829c-24783243a5a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:09.968574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b15cb6-4d6b-4b5a-829c-24783243a5a6 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:09.974230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:09.975238 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:09.975523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:09.975825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] PATH_INFO: `/v3/users/2869f39f6e1b4fc1a41ab5f603132e54` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:09.977307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2869f39f6e1b4fc1a41ab5f603132e54)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:09.979267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:10.010892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:10.025578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2869f39f6e1b4fc1a41ab5f603132e54'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.032259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2869f39f6e1b4fc1a41ab5f603132e54'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.032937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2869f39f6e1b4fc1a41ab5f603132e54'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.045092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2869f39f6e1b4fc1a41ab5f603132e54'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.052627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2869f39f6e1b4fc1a41ab5f603132e54'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.058523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1290/2579] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:10:09 2022] DELETE /identity/v3/users/2869f39f6e1b4fc1a41ab5f603132e54 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:10.063082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451d0f9-e4ff-4347-bd20-fe88ef31ce46 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:10.063566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451d0f9-e4ff-4347-bd20-fe88ef31ce46 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:10.063702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451d0f9-e4ff-4347-bd20-fe88ef31ce46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:10.080397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1451d0f9-e4ff-4347-bd20-fe88ef31ce46 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:10.081581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1451d0f9-e4ff-4347-bd20-fe88ef31ce46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:10.139154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1290/2580] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:10.149542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:10.151154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:10.207871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb86c8b-6f42-4256-b138-c7544206ab94 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:10.212174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109e855-2a81-4467-b29e-e3d9147f7d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:10.212785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109e855-2a81-4467-b29e-e3d9147f7d85 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:10.213067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109e855-2a81-4467-b29e-e3d9147f7d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:10.213346 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109e855-2a81-4467-b29e-e3d9147f7d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:10.223226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109e855-2a81-4467-b29e-e3d9147f7d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:10.225348 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109e855-2a81-4467-b29e-e3d9147f7d85 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:10.234960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1291/2581] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:10.512513 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451d0f9-e4ff-4347-bd20-fe88ef31ce46 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:10.514038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1451d0f9-e4ff-4347-bd20-fe88ef31ce46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:10.526994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451d0f9-e4ff-4347-bd20-fe88ef31ce46 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:10.531801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:10.532795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:10.533143 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:10.533494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] PATH_INFO: `/v3/projects/64e659a8bfef451fb1de70624606be12` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:10.534677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e659a8bfef451fb1de70624606be12)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:10.536897 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:10.582433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e659a8bfef451fb1de70624606be12', 'request_id': 'req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.590259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e659a8bfef451fb1de70624606be12', 'request_id': 'req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.595933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e659a8bfef451fb1de70624606be12', 'request_id': 'req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.607338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] The token cache is being invalidate because project 64e659a8bfef451fb1de70624606be12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:10:10.607628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.608260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.609862 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1291/2582] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:10:10 2022] DELETE /identity/v3/projects/64e659a8bfef451fb1de70624606be12 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:10.613893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109e855-2a81-4467-b29e-e3d9147f7d85 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:10.624060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109e855-2a81-4467-b29e-e3d9147f7d85 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:10.680624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:10.681423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:10.681712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:10.681990 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] PATH_INFO: `/v3/users/4d336f1805084bc5926634b2e7fa3d2d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:10.683166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d336f1805084bc5926634b2e7fa3d2d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:10.685131 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:10.733970 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d336f1805084bc5926634b2e7fa3d2d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.738838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:10.743487 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d336f1805084bc5926634b2e7fa3d2d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.744366 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d336f1805084bc5926634b2e7fa3d2d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.757586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d336f1805084bc5926634b2e7fa3d2d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.765511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-41b8bc05-ac92-4e65-a536-4193d94853be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d336f1805084bc5926634b2e7fa3d2d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:10.772433 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1292/2583] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:10:10 2022] DELETE /identity/v3/users/4d336f1805084bc5926634b2e7fa3d2d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:11.101878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:11.103338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:11.117710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65774-e4bd-4ee4-8229-d6b0beb7fd2b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:11.156700 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:11.182842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:11.183708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.183997 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.184272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] PATH_INFO: `/v3/projects/65bea520bcfb44b4900ec184bf0165c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.185524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65bea520bcfb44b4900ec184bf0165c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:11.187373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:11.233183 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65bea520bcfb44b4900ec184bf0165c7', 'request_id': 'req-832571b5-68e8-48d4-8325-aa159eeebe81'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:11.241147 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65bea520bcfb44b4900ec184bf0165c7', 'request_id': 'req-832571b5-68e8-48d4-8325-aa159eeebe81'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:11.246757 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65bea520bcfb44b4900ec184bf0165c7', 'request_id': 'req-832571b5-68e8-48d4-8325-aa159eeebe81'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:11.257652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] The token cache is being invalidate because project 65bea520bcfb44b4900ec184bf0165c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:10:11.257942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:11.258574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-832571b5-68e8-48d4-8325-aa159eeebe81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:11.260438 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1292/2584] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:10:11 2022] DELETE /identity/v3/projects/65bea520bcfb44b4900ec184bf0165c7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:11.265107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbdd75-fdd0-44b2-8684-e2ac0e3597a9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.265585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbdd75-fdd0-44b2-8684-e2ac0e3597a9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.265663 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbdd75-fdd0-44b2-8684-e2ac0e3597a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.283825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-51cbdd75-fdd0-44b2-8684-e2ac0e3597a9 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:11.285154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-51cbdd75-fdd0-44b2-8684-e2ac0e3597a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:11.357655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1293/2585] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:10:11.366068 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd56586-06c8-469c-8f27-069d2eb50013 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.366650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd56586-06c8-469c-8f27-069d2eb50013 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.366937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd56586-06c8-469c-8f27-069d2eb50013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.385337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3fd56586-06c8-469c-8f27-069d2eb50013 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:11.386546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3fd56586-06c8-469c-8f27-069d2eb50013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:11.395260 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1293/2586] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:11.399969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbdd75-fdd0-44b2-8684-e2ac0e3597a9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:11.409835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbdd75-fdd0-44b2-8684-e2ac0e3597a9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:11.415141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03539e48-735b-40e4-8e07-527cb64e7b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:11.415792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03539e48-735b-40e4-8e07-527cb64e7b70 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.416106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03539e48-735b-40e4-8e07-527cb64e7b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.416379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03539e48-735b-40e4-8e07-527cb64e7b70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.417153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03539e48-735b-40e4-8e07-527cb64e7b70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:10:11.417460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03539e48-735b-40e4-8e07-527cb64e7b70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:11.419050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03539e48-735b-40e4-8e07-527cb64e7b70 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:11.431637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1294/2587] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:10:11.436116 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd56586-06c8-469c-8f27-069d2eb50013 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:11.445383 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd56586-06c8-469c-8f27-069d2eb50013 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:11.450064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d42e8-227a-40db-8b77-cc6772b2634f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:11.450708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d42e8-227a-40db-8b77-cc6772b2634f admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.450985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d42e8-227a-40db-8b77-cc6772b2634f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.451254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d42e8-227a-40db-8b77-cc6772b2634f admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.451939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d42e8-227a-40db-8b77-cc6772b2634f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:11.453868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d42e8-227a-40db-8b77-cc6772b2634f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:11.479644 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1294/2588] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:10:11 2022] POST /identity/v3/projects => generated 409 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:10:11.483632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03539e48-735b-40e4-8e07-527cb64e7b70 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:11.494248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03539e48-735b-40e4-8e07-527cb64e7b70 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:11.549458 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01fbf-3a46-452e-962d-09bdf6731fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:11.550390 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01fbf-3a46-452e-962d-09bdf6731fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.550668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01fbf-3a46-452e-962d-09bdf6731fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.550942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01fbf-3a46-452e-962d-09bdf6731fc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.551630 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b01fbf-3a46-452e-962d-09bdf6731fc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:11.553636 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b01fbf-3a46-452e-962d-09bdf6731fc9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:11.572870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1295/2589] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:10:11 2022] POST /identity/v3/users => generated 305 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:11.576971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d42e8-227a-40db-8b77-cc6772b2634f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:11.587683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d42e8-227a-40db-8b77-cc6772b2634f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:11.592185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d5305-134a-407d-937e-61a2db0ff061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:11.592874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d5305-134a-407d-937e-61a2db0ff061 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.593154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d5305-134a-407d-937e-61a2db0ff061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.593426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d5305-134a-407d-937e-61a2db0ff061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.594144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d5305-134a-407d-937e-61a2db0ff061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:11.595716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d5305-134a-407d-937e-61a2db0ff061 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:11.606598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1295/2590] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:11.610379 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01fbf-3a46-452e-962d-09bdf6731fc9 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:11.620669 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01fbf-3a46-452e-962d-09bdf6731fc9 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:11.625270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f505849-3e32-4b63-b4e1-8879091c89fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:11.625922 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f505849-3e32-4b63-b4e1-8879091c89fc admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.626220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f505849-3e32-4b63-b4e1-8879091c89fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.626481 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f505849-3e32-4b63-b4e1-8879091c89fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.627206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f505849-3e32-4b63-b4e1-8879091c89fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:11.628834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f505849-3e32-4b63-b4e1-8879091c89fc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:11.639298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1296/2591] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:11.643329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d5305-134a-407d-937e-61a2db0ff061 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:11.653901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d5305-134a-407d-937e-61a2db0ff061 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:11.658489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22640-28a0-47ed-b0e3-63fa2fb2b367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:11.659354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22640-28a0-47ed-b0e3-63fa2fb2b367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.659637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22640-28a0-47ed-b0e3-63fa2fb2b367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.659915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22640-28a0-47ed-b0e3-63fa2fb2b367 admin admin] PATH_INFO: `/v3/projects/9b53bf6532e0478cbbe6711edd7c4b0d/users/07746e3d96384a1ab641d48f91d51a07/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.674694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda22640-28a0-47ed-b0e3-63fa2fb2b367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b53bf6532e0478cbbe6711edd7c4b0d, user_id=07746e3d96384a1ab641d48f91d51a07, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:11.677861 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda22640-28a0-47ed-b0e3-63fa2fb2b367 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:11.691089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1296/2592] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:10:11 2022] PUT /identity/v3/projects/9b53bf6532e0478cbbe6711edd7c4b0d/users/07746e3d96384a1ab641d48f91d51a07/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:11.701654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984eb9f-b883-4e23-a0a8-d4ae89fd4e63 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.701972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984eb9f-b883-4e23-a0a8-d4ae89fd4e63 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.702279 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984eb9f-b883-4e23-a0a8-d4ae89fd4e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.729796 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3984eb9f-b883-4e23-a0a8-d4ae89fd4e63 None None] MFA Rules not processed for user `07746e3d96384a1ab641d48f91d51a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:11.731012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3984eb9f-b883-4e23-a0a8-d4ae89fd4e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:11.754319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:11.797286 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1297/2593] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:11 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:11.802716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22640-28a0-47ed-b0e3-63fa2fb2b367 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:11.865034 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22640-28a0-47ed-b0e3-63fa2fb2b367 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:11.869940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcb963-3b8e-47d1-825f-5ba8a1252c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:11.870596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcb963-3b8e-47d1-825f-5ba8a1252c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.870878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcb963-3b8e-47d1-825f-5ba8a1252c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.871163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcb963-3b8e-47d1-825f-5ba8a1252c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.871885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcb963-3b8e-47d1-825f-5ba8a1252c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:11.873810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcb963-3b8e-47d1-825f-5ba8a1252c09 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:11.882992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:11.901624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1297/2594] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:10:11 2022] POST /identity/v3/projects => generated 407 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:11.905505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984eb9f-b883-4e23-a0a8-d4ae89fd4e63 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:11.917060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984eb9f-b883-4e23-a0a8-d4ae89fd4e63 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:11.974341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e69a0-7dbd-4e52-9c54-cf89f1aa3502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:11.974930 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e69a0-7dbd-4e52-9c54-cf89f1aa3502 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:11.975214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e69a0-7dbd-4e52-9c54-cf89f1aa3502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:11.975489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e69a0-7dbd-4e52-9c54-cf89f1aa3502 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:11.976178 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e69a0-7dbd-4e52-9c54-cf89f1aa3502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:11.978115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e69a0-7dbd-4e52-9c54-cf89f1aa3502 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:11.997142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1298/2595] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:10:11 2022] POST /identity/v3/users => generated 310 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:12.001092 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcb963-3b8e-47d1-825f-5ba8a1252c09 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:12.011782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcb963-3b8e-47d1-825f-5ba8a1252c09 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:12.016456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ebbe4-884a-4d1d-ab25-5f57545ec6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:12.017121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ebbe4-884a-4d1d-ab25-5f57545ec6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:12.017400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ebbe4-884a-4d1d-ab25-5f57545ec6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:12.017671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ebbe4-884a-4d1d-ab25-5f57545ec6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:12.018631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ebbe4-884a-4d1d-ab25-5f57545ec6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:12.020128 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ebbe4-884a-4d1d-ab25-5f57545ec6d2 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:12.036062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1298/2596] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:12.037125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e69a0-7dbd-4e52-9c54-cf89f1aa3502 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:12.047538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e69a0-7dbd-4e52-9c54-cf89f1aa3502 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:12.052490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43653f-cd87-499f-baee-25ce73cfae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:12.053362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43653f-cd87-499f-baee-25ce73cfae1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:12.053649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43653f-cd87-499f-baee-25ce73cfae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:12.053921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43653f-cd87-499f-baee-25ce73cfae1e admin admin] PATH_INFO: `/v3/projects/af4e3c3d3351428a87600708ef355e18/users/5981d9c5db684c79ab9bb8f8fe28cbe7/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:12.078400 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43653f-cd87-499f-baee-25ce73cfae1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4e3c3d3351428a87600708ef355e18, user_id=5981d9c5db684c79ab9bb8f8fe28cbe7, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:12.081179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43653f-cd87-499f-baee-25ce73cfae1e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:12.095595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1299/2597] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:10:12 2022] PUT /identity/v3/projects/af4e3c3d3351428a87600708ef355e18/users/5981d9c5db684c79ab9bb8f8fe28cbe7/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:10:12.099744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ebbe4-884a-4d1d-ab25-5f57545ec6d2 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:12.110305 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ebbe4-884a-4d1d-ab25-5f57545ec6d2 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:12.170149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d0275-1d4a-425a-8681-8eb4445df90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:12.170843 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d0275-1d4a-425a-8681-8eb4445df90a admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:12.171149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d0275-1d4a-425a-8681-8eb4445df90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:12.171420 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d0275-1d4a-425a-8681-8eb4445df90a admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:12.172134 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d0275-1d4a-425a-8681-8eb4445df90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:12.173748 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d0275-1d4a-425a-8681-8eb4445df90a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:12.184342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1299/2598] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:12.188568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43653f-cd87-499f-baee-25ce73cfae1e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:12.199134 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43653f-cd87-499f-baee-25ce73cfae1e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:12.203896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137915ce-060f-4bf3-a852-832f0fcec188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:12.204806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137915ce-060f-4bf3-a852-832f0fcec188 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:12.205088 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137915ce-060f-4bf3-a852-832f0fcec188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:12.205359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137915ce-060f-4bf3-a852-832f0fcec188 admin admin] PATH_INFO: `/v3/projects/af4e3c3d3351428a87600708ef355e18/users/5981d9c5db684c79ab9bb8f8fe28cbe7/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:12.207324 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137915ce-060f-4bf3-a852-832f0fcec188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4e3c3d3351428a87600708ef355e18, user_id=5981d9c5db684c79ab9bb8f8fe28cbe7, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:12.210583 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137915ce-060f-4bf3-a852-832f0fcec188 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:12.222590 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1300/2599] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:10:12 2022] PUT /identity/v3/projects/af4e3c3d3351428a87600708ef355e18/users/5981d9c5db684c79ab9bb8f8fe28cbe7/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:12.234039 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e235576-ccad-4b2a-b612-4edcff5b0a1f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:12.234339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e235576-ccad-4b2a-b612-4edcff5b0a1f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:12.234625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e235576-ccad-4b2a-b612-4edcff5b0a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:12.263199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-9e235576-ccad-4b2a-b612-4edcff5b0a1f None None] MFA Rules not processed for user `5981d9c5db684c79ab9bb8f8fe28cbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:12.264475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9e235576-ccad-4b2a-b612-4edcff5b0a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:12.334189 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1300/2600] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:12 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:12.344723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137915ce-060f-4bf3-a852-832f0fcec188 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:12.346547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-137915ce-060f-4bf3-a852-832f0fcec188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:12.403967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137915ce-060f-4bf3-a852-832f0fcec188 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:12.407605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fed9be-6b5e-4c1d-886c-d60adfb5f3b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:12.408239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fed9be-6b5e-4c1d-886c-d60adfb5f3b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:12.408562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fed9be-6b5e-4c1d-886c-d60adfb5f3b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:12.408841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fed9be-6b5e-4c1d-886c-d60adfb5f3b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:12.418258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fed9be-6b5e-4c1d-886c-d60adfb5f3b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:12.419686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fed9be-6b5e-4c1d-886c-d60adfb5f3b7 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:12.429637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1301/2601] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:12 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:12.579121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e235576-ccad-4b2a-b612-4edcff5b0a1f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:12.588668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e235576-ccad-4b2a-b612-4edcff5b0a1f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:12.592151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ac760-3990-418a-9b0c-d829ec1dffe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:12.592820 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ac760-3990-418a-9b0c-d829ec1dffe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:12.593097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ac760-3990-418a-9b0c-d829ec1dffe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:12.593368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ac760-3990-418a-9b0c-d829ec1dffe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:12.648825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ac760-3990-418a-9b0c-d829ec1dffe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:12.650267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ac760-3990-418a-9b0c-d829ec1dffe5 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:12.653761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:12.661051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1301/2602] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:19.417846 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fed9be-6b5e-4c1d-886c-d60adfb5f3b7 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:19.419283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-03fed9be-6b5e-4c1d-886c-d60adfb5f3b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:19.444433 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:19.491639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fed9be-6b5e-4c1d-886c-d60adfb5f3b7 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:19.496202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:19.497120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:19.497560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:19.497843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] PATH_INFO: `/v3/users/ed9ac6c572d34bb9bfb930cece53f337` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:19.499062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed9ac6c572d34bb9bfb930cece53f337)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:19.501066 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:19.552860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed9ac6c572d34bb9bfb930cece53f337'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:19.559795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed9ac6c572d34bb9bfb930cece53f337'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:19.560542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed9ac6c572d34bb9bfb930cece53f337'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:19.573509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed9ac6c572d34bb9bfb930cece53f337'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:19.582143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed9ac6c572d34bb9bfb930cece53f337'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:19.589341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1302/2603] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:10:19 2022] DELETE /identity/v3/users/ed9ac6c572d34bb9bfb930cece53f337 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:19.779111 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ac760-3990-418a-9b0c-d829ec1dffe5 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:19.793271 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ac760-3990-418a-9b0c-d829ec1dffe5 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:19.859331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:19.860195 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:19.860510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:19.860787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] PATH_INFO: `/v3/projects/1f7bab4d926040e3a63aa5c35443977a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:19.861952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7bab4d926040e3a63aa5c35443977a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:19.863838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:19.909378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7bab4d926040e3a63aa5c35443977a', 'request_id': 'req-2fcc8257-0862-4c6c-9795-dad3557daa9f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:19.920290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7bab4d926040e3a63aa5c35443977a', 'request_id': 'req-2fcc8257-0862-4c6c-9795-dad3557daa9f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:19.926340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7bab4d926040e3a63aa5c35443977a', 'request_id': 'req-2fcc8257-0862-4c6c-9795-dad3557daa9f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:19.938274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] The token cache is being invalidate because project 1f7bab4d926040e3a63aa5c35443977a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:10:19.938567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:19.939215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:19.940791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1302/2604] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:10:19 2022] DELETE /identity/v3/projects/1f7bab4d926040e3a63aa5c35443977a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:19.945202 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:19.946607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:19.959587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1ad85-d508-4f67-89d4-fea99ff0631d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:20.015579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:20.016476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.016762 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.017043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] PATH_INFO: `/v3/users/cdbc56c254864795894b3298faf53ca0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.018227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdbc56c254864795894b3298faf53ca0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:20.020239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:20.063425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:20.070072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdbc56c254864795894b3298faf53ca0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:20.079380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdbc56c254864795894b3298faf53ca0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:20.080141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdbc56c254864795894b3298faf53ca0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:20.092505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdbc56c254864795894b3298faf53ca0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:20.100048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a6e7a811-23e2-4d6f-9ddd-d97fbaef76a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdbc56c254864795894b3298faf53ca0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:20.106617 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1303/2605] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:10:19 2022] DELETE /identity/v3/users/cdbc56c254864795894b3298faf53ca0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:20.287708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:20.289347 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:20.305931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc8257-0862-4c6c-9795-dad3557daa9f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:20.363870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:20.365325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.365606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.365880 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] PATH_INFO: `/v3/projects/dc4077e272394552ad59e5624a63a351` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.367100 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc4077e272394552ad59e5624a63a351)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:20.369082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:20.401366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:20.415915 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc4077e272394552ad59e5624a63a351', 'request_id': 'req-7925d2db-2bc7-4446-90dc-75c5a92715b6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:20.422740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc4077e272394552ad59e5624a63a351', 'request_id': 'req-7925d2db-2bc7-4446-90dc-75c5a92715b6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:20.429886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc4077e272394552ad59e5624a63a351', 'request_id': 'req-7925d2db-2bc7-4446-90dc-75c5a92715b6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:20.441037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] The token cache is being invalidate because project dc4077e272394552ad59e5624a63a351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:10:20.441325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:20.442081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7925d2db-2bc7-4446-90dc-75c5a92715b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:20.443788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1303/2606] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:10:20 2022] DELETE /identity/v3/projects/dc4077e272394552ad59e5624a63a351 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:20.448700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f905c9-832a-4b13-818b-1626c075cc62 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.449167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f905c9-832a-4b13-818b-1626c075cc62 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.449320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f905c9-832a-4b13-818b-1626c075cc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.466458 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-47f905c9-832a-4b13-818b-1626c075cc62 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:20.467691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-47f905c9-832a-4b13-818b-1626c075cc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:20.526844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1304/2607] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:20.534866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab97fe1-5406-4af0-abfb-b99fea956d21 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.535153 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab97fe1-5406-4af0-abfb-b99fea956d21 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.535427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab97fe1-5406-4af0-abfb-b99fea956d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.554498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-9ab97fe1-5406-4af0-abfb-b99fea956d21 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:20.555677 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9ab97fe1-5406-4af0-abfb-b99fea956d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:20.563454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1304/2608] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:20.567668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f905c9-832a-4b13-818b-1626c075cc62 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:20.577250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f905c9-832a-4b13-818b-1626c075cc62 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:20.581811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906d2ce-6791-4d03-a12d-3ada4527a3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:20.582459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906d2ce-6791-4d03-a12d-3ada4527a3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.583372 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906d2ce-6791-4d03-a12d-3ada4527a3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.583654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906d2ce-6791-4d03-a12d-3ada4527a3d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.584412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906d2ce-6791-4d03-a12d-3ada4527a3d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:10:20.584744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906d2ce-6791-4d03-a12d-3ada4527a3d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:20.586340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906d2ce-6791-4d03-a12d-3ada4527a3d6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:20.597790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1305/2609] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 18:10:20.602525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab97fe1-5406-4af0-abfb-b99fea956d21 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:20.611807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab97fe1-5406-4af0-abfb-b99fea956d21 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:20.616687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c7416-adb6-4b80-838f-26e1f794d1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:20.617342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c7416-adb6-4b80-838f-26e1f794d1ae admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.617627 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c7416-adb6-4b80-838f-26e1f794d1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.617908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c7416-adb6-4b80-838f-26e1f794d1ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.618601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c7416-adb6-4b80-838f-26e1f794d1ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:20.620546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c7416-adb6-4b80-838f-26e1f794d1ae admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:20.645721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1305/2610] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:10:20 2022] POST /identity/v3/projects => generated 363 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:20.649923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906d2ce-6791-4d03-a12d-3ada4527a3d6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:20.659771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906d2ce-6791-4d03-a12d-3ada4527a3d6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:20.716051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ff23-952e-4bb8-a8a8-7f870c204d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:20.716747 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ff23-952e-4bb8-a8a8-7f870c204d51 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.717033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ff23-952e-4bb8-a8a8-7f870c204d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.717315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ff23-952e-4bb8-a8a8-7f870c204d51 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.718009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13ff23-952e-4bb8-a8a8-7f870c204d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:20.719934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13ff23-952e-4bb8-a8a8-7f870c204d51 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:20.740317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1306/2611] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:10:20 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:20.744418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c7416-adb6-4b80-838f-26e1f794d1ae admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:20.754710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c7416-adb6-4b80-838f-26e1f794d1ae admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:20.759740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e5dd0-5db7-4b08-8bde-7cfd32165ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:20.760448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e5dd0-5db7-4b08-8bde-7cfd32165ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.760738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e5dd0-5db7-4b08-8bde-7cfd32165ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.761020 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e5dd0-5db7-4b08-8bde-7cfd32165ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.761785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e5dd0-5db7-4b08-8bde-7cfd32165ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:20.763400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e5dd0-5db7-4b08-8bde-7cfd32165ac3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:20.776216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1306/2612] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:20.778743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ff23-952e-4bb8-a8a8-7f870c204d51 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:20.789691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ff23-952e-4bb8-a8a8-7f870c204d51 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:20.794341 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e18801-5cfb-4f10-8b63-7c2ef4441526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:20.795007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e18801-5cfb-4f10-8b63-7c2ef4441526 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.795293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e18801-5cfb-4f10-8b63-7c2ef4441526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.795568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e18801-5cfb-4f10-8b63-7c2ef4441526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.796317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e18801-5cfb-4f10-8b63-7c2ef4441526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:20.797915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e18801-5cfb-4f10-8b63-7c2ef4441526 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:20.808392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1307/2613] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:20.812510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e5dd0-5db7-4b08-8bde-7cfd32165ac3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:20.822536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e5dd0-5db7-4b08-8bde-7cfd32165ac3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:20.827487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e161f9d-0ea8-46c7-8770-8ecc30b03a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:20.828492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e161f9d-0ea8-46c7-8770-8ecc30b03a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.828770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e161f9d-0ea8-46c7-8770-8ecc30b03a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.829041 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e161f9d-0ea8-46c7-8770-8ecc30b03a10 admin admin] PATH_INFO: `/v3/projects/24c58716f06042efbcc3c9b4da0fb07d/users/0b1ca26e28cf400ab886665c3f4f6e4e/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.844168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e161f9d-0ea8-46c7-8770-8ecc30b03a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c58716f06042efbcc3c9b4da0fb07d, user_id=0b1ca26e28cf400ab886665c3f4f6e4e, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:20.846831 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e161f9d-0ea8-46c7-8770-8ecc30b03a10 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:20.860785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1307/2614] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:10:20 2022] PUT /identity/v3/projects/24c58716f06042efbcc3c9b4da0fb07d/users/0b1ca26e28cf400ab886665c3f4f6e4e/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:20.872465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fa4f4-bb72-4b7f-8ce9-2a0dbb069465 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:20.872763 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fa4f4-bb72-4b7f-8ce9-2a0dbb069465 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:20.873039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fa4f4-bb72-4b7f-8ce9-2a0dbb069465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:20.901240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bf4fa4f4-bb72-4b7f-8ce9-2a0dbb069465 None None] MFA Rules not processed for user `0b1ca26e28cf400ab886665c3f4f6e4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:20.902440 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bf4fa4f4-bb72-4b7f-8ce9-2a0dbb069465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:20.914640 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:20.976262 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1308/2615] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:20.987919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e161f9d-0ea8-46c7-8770-8ecc30b03a10 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:20.989804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6e161f9d-0ea8-46c7-8770-8ecc30b03a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:21.036770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fa4f4-bb72-4b7f-8ce9-2a0dbb069465 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:21.038318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bf4fa4f4-bb72-4b7f-8ce9-2a0dbb069465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:21.051623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e161f9d-0ea8-46c7-8770-8ecc30b03a10 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:21.059370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6c029-66c3-4634-bfb8-eb142d26046e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:21.060024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6c029-66c3-4634-bfb8-eb142d26046e service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:21.060329 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6c029-66c3-4634-bfb8-eb142d26046e service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:21.060624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6c029-66c3-4634-bfb8-eb142d26046e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:21.070323 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6c029-66c3-4634-bfb8-eb142d26046e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:21.072033 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6c029-66c3-4634-bfb8-eb142d26046e service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:21.081858 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1308/2616] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:20 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:21.105907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fa4f4-bb72-4b7f-8ce9-2a0dbb069465 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:21.110416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:21.111275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:21.111557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:21.111834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] PATH_INFO: `/v3/users/07746e3d96384a1ab641d48f91d51a07` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:21.113047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07746e3d96384a1ab641d48f91d51a07)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:21.115655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:21.166650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07746e3d96384a1ab641d48f91d51a07'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.173848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07746e3d96384a1ab641d48f91d51a07'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.174533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07746e3d96384a1ab641d48f91d51a07'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.187067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07746e3d96384a1ab641d48f91d51a07'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.196102 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07746e3d96384a1ab641d48f91d51a07'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.203049 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1309/2617] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:10:21 2022] DELETE /identity/v3/users/07746e3d96384a1ab641d48f91d51a07 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:21.210085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9782b-357a-4a21-869b-a4dd542b9726 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:21.210406 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9782b-357a-4a21-869b-a4dd542b9726 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:21.210705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9782b-357a-4a21-869b-a4dd542b9726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:21.228549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-99e9782b-357a-4a21-869b-a4dd542b9726 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:21.229978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-99e9782b-357a-4a21-869b-a4dd542b9726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:21.287976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:21.298429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1309/2618] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:21.309431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:21.311222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:21.350723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:21.377039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8a979-c421-4131-988f-89ec7f6995f3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:21.384416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acfa94-b49c-4b53-b6aa-28d0bf0d78a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:21.385407 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acfa94-b49c-4b53-b6aa-28d0bf0d78a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:21.385517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acfa94-b49c-4b53-b6aa-28d0bf0d78a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:21.385841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acfa94-b49c-4b53-b6aa-28d0bf0d78a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:21.395869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acfa94-b49c-4b53-b6aa-28d0bf0d78a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:21.397329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acfa94-b49c-4b53-b6aa-28d0bf0d78a1 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:21.407133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1310/2619] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:21.679030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9782b-357a-4a21-869b-a4dd542b9726 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:21.680517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-99e9782b-357a-4a21-869b-a4dd542b9726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:21.693196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9782b-357a-4a21-869b-a4dd542b9726 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:21.697797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:21.698694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:21.698982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:21.699261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] PATH_INFO: `/v3/projects/9b53bf6532e0478cbbe6711edd7c4b0d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:21.700489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b53bf6532e0478cbbe6711edd7c4b0d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:21.702435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:21.751665 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b53bf6532e0478cbbe6711edd7c4b0d', 'request_id': 'req-b6a5a592-16a3-4a74-91db-4019826cf164'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.759161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b53bf6532e0478cbbe6711edd7c4b0d', 'request_id': 'req-b6a5a592-16a3-4a74-91db-4019826cf164'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.765465 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b53bf6532e0478cbbe6711edd7c4b0d', 'request_id': 'req-b6a5a592-16a3-4a74-91db-4019826cf164'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.776534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] The token cache is being invalidate because project 9b53bf6532e0478cbbe6711edd7c4b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:10:21.776815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.777463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.779216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1310/2620] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:10:21 2022] DELETE /identity/v3/projects/9b53bf6532e0478cbbe6711edd7c4b0d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:21.783230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acfa94-b49c-4b53-b6aa-28d0bf0d78a1 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:21.793330 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acfa94-b49c-4b53-b6aa-28d0bf0d78a1 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:21.852527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:21.853345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:21.853622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:21.853892 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] PATH_INFO: `/v3/users/5981d9c5db684c79ab9bb8f8fe28cbe7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:21.855073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5981d9c5db684c79ab9bb8f8fe28cbe7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:21.857023 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:21.902576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5981d9c5db684c79ab9bb8f8fe28cbe7'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.909298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5981d9c5db684c79ab9bb8f8fe28cbe7'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.909951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5981d9c5db684c79ab9bb8f8fe28cbe7'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.922207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5981d9c5db684c79ab9bb8f8fe28cbe7'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.930116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bd20376a-28e8-4609-aa3a-2cfddc03af48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5981d9c5db684c79ab9bb8f8fe28cbe7'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:21.936251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1311/2621] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:10:21 2022] DELETE /identity/v3/users/5981d9c5db684c79ab9bb8f8fe28cbe7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:22.216430 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:22.217878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:22.235194 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5a592-16a3-4a74-91db-4019826cf164 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:22.292764 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:22.293624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:22.293901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:22.294172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] PATH_INFO: `/v3/projects/af4e3c3d3351428a87600708ef355e18` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:22.295357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] RBAC: Authorizing `identity:delete_project(project_id=af4e3c3d3351428a87600708ef355e18)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:22.297290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:22.342480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af4e3c3d3351428a87600708ef355e18', 'request_id': 'req-64d4c371-f35b-448e-b488-14cf91d7523a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:22.350207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af4e3c3d3351428a87600708ef355e18', 'request_id': 'req-64d4c371-f35b-448e-b488-14cf91d7523a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:22.355739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af4e3c3d3351428a87600708ef355e18', 'request_id': 'req-64d4c371-f35b-448e-b488-14cf91d7523a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:22.366556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] The token cache is being invalidate because project af4e3c3d3351428a87600708ef355e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:10:22.366841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:22.367471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-64d4c371-f35b-448e-b488-14cf91d7523a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:22.369073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1311/2622] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:10:22 2022] DELETE /identity/v3/projects/af4e3c3d3351428a87600708ef355e18 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:22.374022 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e998f-5995-4705-ac3e-11026aa9f49f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:22.374315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e998f-5995-4705-ac3e-11026aa9f49f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:22.374626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e998f-5995-4705-ac3e-11026aa9f49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:22.392652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e28e998f-5995-4705-ac3e-11026aa9f49f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:22.393840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e28e998f-5995-4705-ac3e-11026aa9f49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:22.448492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:22.462745 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1312/2623] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:22.470624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027eb1a-2bc8-436b-9a4e-c320cfdd1692 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:22.470911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027eb1a-2bc8-436b-9a4e-c320cfdd1692 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:22.471187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027eb1a-2bc8-436b-9a4e-c320cfdd1692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:22.492364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2027eb1a-2bc8-436b-9a4e-c320cfdd1692 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:22.493596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2027eb1a-2bc8-436b-9a4e-c320cfdd1692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:22.501709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1312/2624] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:22.506939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e998f-5995-4705-ac3e-11026aa9f49f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:22.517228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e998f-5995-4705-ac3e-11026aa9f49f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:22.523944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964348c-e8e8-4241-9d99-a219a81b6769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:22.525016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964348c-e8e8-4241-9d99-a219a81b6769 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:22.525413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964348c-e8e8-4241-9d99-a219a81b6769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:22.525723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964348c-e8e8-4241-9d99-a219a81b6769 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:22.526595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964348c-e8e8-4241-9d99-a219a81b6769 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:10:22.526936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964348c-e8e8-4241-9d99-a219a81b6769 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:22.528681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964348c-e8e8-4241-9d99-a219a81b6769 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:22.544329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1313/2625] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:10:22.548583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027eb1a-2bc8-436b-9a4e-c320cfdd1692 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:22.555930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:22.560090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027eb1a-2bc8-436b-9a4e-c320cfdd1692 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:22.566239 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08067d42-6895-4519-a7ab-f2ce4307df1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:22.567087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08067d42-6895-4519-a7ab-f2ce4307df1e admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:22.567622 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08067d42-6895-4519-a7ab-f2ce4307df1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:22.568080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08067d42-6895-4519-a7ab-f2ce4307df1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:22.568983 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08067d42-6895-4519-a7ab-f2ce4307df1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:22.571080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08067d42-6895-4519-a7ab-f2ce4307df1e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:22.600904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1313/2626] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:10:22 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:22.602097 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964348c-e8e8-4241-9d99-a219a81b6769 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:22.612376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964348c-e8e8-4241-9d99-a219a81b6769 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:22.671444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f747e27-c37a-426f-bf71-16259efbf09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:22.672167 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f747e27-c37a-426f-bf71-16259efbf09b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:22.672478 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f747e27-c37a-426f-bf71-16259efbf09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:22.672754 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f747e27-c37a-426f-bf71-16259efbf09b admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:22.673472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f747e27-c37a-426f-bf71-16259efbf09b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:22.675596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f747e27-c37a-426f-bf71-16259efbf09b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:22.698157 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1314/2627] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:10:22 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:22.702342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08067d42-6895-4519-a7ab-f2ce4307df1e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:22.712663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08067d42-6895-4519-a7ab-f2ce4307df1e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:22.717224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba26736-6a7d-4a44-a13a-245d013bbdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:22.717879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba26736-6a7d-4a44-a13a-245d013bbdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:22.718164 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba26736-6a7d-4a44-a13a-245d013bbdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:22.718437 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba26736-6a7d-4a44-a13a-245d013bbdf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:22.719160 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba26736-6a7d-4a44-a13a-245d013bbdf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:22.720766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba26736-6a7d-4a44-a13a-245d013bbdf9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:22.732066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1314/2628] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:22.736254 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f747e27-c37a-426f-bf71-16259efbf09b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:22.747009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f747e27-c37a-426f-bf71-16259efbf09b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:22.751872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b06e52-ea81-43dc-ba3f-3a8f52a3e04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:22.752704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b06e52-ea81-43dc-ba3f-3a8f52a3e04b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:22.753080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b06e52-ea81-43dc-ba3f-3a8f52a3e04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:22.753359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b06e52-ea81-43dc-ba3f-3a8f52a3e04b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:22.754085 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b06e52-ea81-43dc-ba3f-3a8f52a3e04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:22.755773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b06e52-ea81-43dc-ba3f-3a8f52a3e04b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:22.767111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1315/2629] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 18:10:22.771947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba26736-6a7d-4a44-a13a-245d013bbdf9 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:22.782013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba26736-6a7d-4a44-a13a-245d013bbdf9 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:22.787789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952f3c8-6569-418b-963d-4b2225e4ea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:22.788837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952f3c8-6569-418b-963d-4b2225e4ea39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:22.789217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952f3c8-6569-418b-963d-4b2225e4ea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:22.789640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952f3c8-6569-418b-963d-4b2225e4ea39 admin admin] PATH_INFO: `/v3/projects/be80b208dfae41ccb05e21e9d2a685a6/users/3e1a3fc71208421f86c3d69d28a21878/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:22.805715 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952f3c8-6569-418b-963d-4b2225e4ea39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be80b208dfae41ccb05e21e9d2a685a6, user_id=3e1a3fc71208421f86c3d69d28a21878, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:22.808451 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952f3c8-6569-418b-963d-4b2225e4ea39 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:22.823503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1315/2630] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:10:22 2022] PUT /identity/v3/projects/be80b208dfae41ccb05e21e9d2a685a6/users/3e1a3fc71208421f86c3d69d28a21878/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:22.836483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b74b-bb2b-481f-abe2-748238e0527d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:22.836979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b74b-bb2b-481f-abe2-748238e0527d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:22.837214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b74b-bb2b-481f-abe2-748238e0527d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:22.870556 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5765b74b-bb2b-481f-abe2-748238e0527d None None] MFA Rules not processed for user `3e1a3fc71208421f86c3d69d28a21878`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:22.871957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5765b74b-bb2b-481f-abe2-748238e0527d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:22.953851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1316/2631] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:22.958877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952f3c8-6569-418b-963d-4b2225e4ea39 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:23.021630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952f3c8-6569-418b-963d-4b2225e4ea39 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:23.026283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad0006-00f3-4216-ae41-51f04c01af19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:23.026942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad0006-00f3-4216-ae41-51f04c01af19 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:23.027226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad0006-00f3-4216-ae41-51f04c01af19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:23.027501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad0006-00f3-4216-ae41-51f04c01af19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:23.028196 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad0006-00f3-4216-ae41-51f04c01af19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:23.030166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad0006-00f3-4216-ae41-51f04c01af19 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:23.055777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1316/2632] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:10:22 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:23.059810 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765b74b-bb2b-481f-abe2-748238e0527d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:23.070662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765b74b-bb2b-481f-abe2-748238e0527d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:23.117394 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:23.128080 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6a2de-5f0d-4d86-9b40-527cbd3b3914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:23.128841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6a2de-5f0d-4d86-9b40-527cbd3b3914 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:23.129128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6a2de-5f0d-4d86-9b40-527cbd3b3914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:23.129900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6a2de-5f0d-4d86-9b40-527cbd3b3914 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:23.130838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6a2de-5f0d-4d86-9b40-527cbd3b3914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:23.132891 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6a2de-5f0d-4d86-9b40-527cbd3b3914 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:23.153301 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1317/2633] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:10:23 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:23.157282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad0006-00f3-4216-ae41-51f04c01af19 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:23.167780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad0006-00f3-4216-ae41-51f04c01af19 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:23.172396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c813f1-1878-48e8-99d3-a235eb4405fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:23.173060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c813f1-1878-48e8-99d3-a235eb4405fa admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:23.173358 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c813f1-1878-48e8-99d3-a235eb4405fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:23.173642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c813f1-1878-48e8-99d3-a235eb4405fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:23.174398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c813f1-1878-48e8-99d3-a235eb4405fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:23.176257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c813f1-1878-48e8-99d3-a235eb4405fa admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:23.186977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1317/2634] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:23.191063 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6a2de-5f0d-4d86-9b40-527cbd3b3914 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:23.201602 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6a2de-5f0d-4d86-9b40-527cbd3b3914 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:23.206154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7013bc7-f5a7-4559-8754-8f402e3c9c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:23.207026 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7013bc7-f5a7-4559-8754-8f402e3c9c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:23.207311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7013bc7-f5a7-4559-8754-8f402e3c9c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:23.208285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7013bc7-f5a7-4559-8754-8f402e3c9c44 admin admin] PATH_INFO: `/v3/projects/a5352f05baa64793b2278682382acdf9/users/b1bde021bba94313b0fdce6ed387bbc1/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:23.222654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7013bc7-f5a7-4559-8754-8f402e3c9c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5352f05baa64793b2278682382acdf9, user_id=b1bde021bba94313b0fdce6ed387bbc1, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:23.225362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7013bc7-f5a7-4559-8754-8f402e3c9c44 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:23.238361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1318/2635] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:10:23 2022] PUT /identity/v3/projects/a5352f05baa64793b2278682382acdf9/users/b1bde021bba94313b0fdce6ed387bbc1/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:23.242405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c813f1-1878-48e8-99d3-a235eb4405fa admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:23.252240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c813f1-1878-48e8-99d3-a235eb4405fa admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:23.308319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f56a32-9c38-4058-ad93-60f3b8b3ca8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:23.309037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f56a32-9c38-4058-ad93-60f3b8b3ca8d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:23.309400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f56a32-9c38-4058-ad93-60f3b8b3ca8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:23.309689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f56a32-9c38-4058-ad93-60f3b8b3ca8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:23.310415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f56a32-9c38-4058-ad93-60f3b8b3ca8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:23.311989 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f56a32-9c38-4058-ad93-60f3b8b3ca8d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:23.322497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1318/2636] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:23.327164 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7013bc7-f5a7-4559-8754-8f402e3c9c44 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:23.338537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7013bc7-f5a7-4559-8754-8f402e3c9c44 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:23.343061 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb74de4-81a3-4358-9843-c2b67750b1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:23.343933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb74de4-81a3-4358-9843-c2b67750b1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:23.344215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb74de4-81a3-4358-9843-c2b67750b1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:23.344527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb74de4-81a3-4358-9843-c2b67750b1f5 admin admin] PATH_INFO: `/v3/projects/a5352f05baa64793b2278682382acdf9/users/b1bde021bba94313b0fdce6ed387bbc1/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:23.346522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb74de4-81a3-4358-9843-c2b67750b1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5352f05baa64793b2278682382acdf9, user_id=b1bde021bba94313b0fdce6ed387bbc1, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:23.349603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb74de4-81a3-4358-9843-c2b67750b1f5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:23.359684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1319/2637] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:10:23 2022] PUT /identity/v3/projects/a5352f05baa64793b2278682382acdf9/users/b1bde021bba94313b0fdce6ed387bbc1/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:10:23.370339 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d8ae1-2da9-4bac-919e-37dc99c63f43 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:23.370714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d8ae1-2da9-4bac-919e-37dc99c63f43 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:23.371076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d8ae1-2da9-4bac-919e-37dc99c63f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:23.399216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b29d8ae1-2da9-4bac-919e-37dc99c63f43 None None] MFA Rules not processed for user `b1bde021bba94313b0fdce6ed387bbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:23.400450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b29d8ae1-2da9-4bac-919e-37dc99c63f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:23.405453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:23.469195 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1319/2638] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:23.480313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb74de4-81a3-4358-9843-c2b67750b1f5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:23.482031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-beb74de4-81a3-4358-9843-c2b67750b1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:23.538196 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb74de4-81a3-4358-9843-c2b67750b1f5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:23.541771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61583c10-8d70-42e5-8779-656d6b31e922 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:23.542397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61583c10-8d70-42e5-8779-656d6b31e922 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:23.542677 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61583c10-8d70-42e5-8779-656d6b31e922 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:23.542951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61583c10-8d70-42e5-8779-656d6b31e922 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:23.597989 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61583c10-8d70-42e5-8779-656d6b31e922 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:23.599565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61583c10-8d70-42e5-8779-656d6b31e922 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:23.608586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1320/2639] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:27.816644 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d8ae1-2da9-4bac-919e-37dc99c63f43 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:27.818674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b29d8ae1-2da9-4bac-919e-37dc99c63f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:27.830959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d8ae1-2da9-4bac-919e-37dc99c63f43 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:27.835141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1489f1b-a5ae-4e7a-b598-ac58f242fd95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:27.836724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1489f1b-a5ae-4e7a-b598-ac58f242fd95 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:27.837082 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1489f1b-a5ae-4e7a-b598-ac58f242fd95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:27.837766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1489f1b-a5ae-4e7a-b598-ac58f242fd95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:27.852640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1489f1b-a5ae-4e7a-b598-ac58f242fd95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:27.852640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1489f1b-a5ae-4e7a-b598-ac58f242fd95 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:27.861501 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1320/2640] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:33.604516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:34.414661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61583c10-8d70-42e5-8779-656d6b31e922 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:34.416107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-61583c10-8d70-42e5-8779-656d6b31e922 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:34.475005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61583c10-8d70-42e5-8779-656d6b31e922 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:34.478578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbccae-df1e-44f1-bcbe-24c4dcff881d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:34.479221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbccae-df1e-44f1-bcbe-24c4dcff881d service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:34.479507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbccae-df1e-44f1-bcbe-24c4dcff881d service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:34.479788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbccae-df1e-44f1-bcbe-24c4dcff881d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:34.481547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1fdbccae-df1e-44f1-bcbe-24c4dcff881d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:34.490972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbccae-df1e-44f1-bcbe-24c4dcff881d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:34.492434 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbccae-df1e-44f1-bcbe-24c4dcff881d service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:34.500311 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1321/2641] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:10:35.952037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1489f1b-a5ae-4e7a-b598-ac58f242fd95 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:36.016597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1489f1b-a5ae-4e7a-b598-ac58f242fd95 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:36.021768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:36.022631 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:36.022909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:36.023179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] PATH_INFO: `/v3/users/3e1a3fc71208421f86c3d69d28a21878` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:36.024391 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e1a3fc71208421f86c3d69d28a21878)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:36.026343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:36.077898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e1a3fc71208421f86c3d69d28a21878'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.078551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e1a3fc71208421f86c3d69d28a21878'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.079210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e1a3fc71208421f86c3d69d28a21878'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.087248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e1a3fc71208421f86c3d69d28a21878'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.101073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e1a3fc71208421f86c3d69d28a21878'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.114461 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1321/2642] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:10:35 2022] DELETE /identity/v3/users/3e1a3fc71208421f86c3d69d28a21878 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:36.119500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281a100-cd58-4160-b052-ee8d9c3cb001 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:36.119942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281a100-cd58-4160-b052-ee8d9c3cb001 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:36.120329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281a100-cd58-4160-b052-ee8d9c3cb001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:36.137544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9281a100-cd58-4160-b052-ee8d9c3cb001 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:36.138825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9281a100-cd58-4160-b052-ee8d9c3cb001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:36.201838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1322/2643] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:36.212782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:36.214025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:36.228892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:36.287521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006da42a-3630-4239-b58b-3957dffedb96 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:36.291207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfc639-16a0-4a9c-88bd-aefcacf8b2e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:36.291863 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfc639-16a0-4a9c-88bd-aefcacf8b2e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:36.292148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfc639-16a0-4a9c-88bd-aefcacf8b2e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:36.292603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfc639-16a0-4a9c-88bd-aefcacf8b2e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:36.303225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfc639-16a0-4a9c-88bd-aefcacf8b2e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:36.304710 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfc639-16a0-4a9c-88bd-aefcacf8b2e3 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:36.316481 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1322/2644] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:36.591230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281a100-cd58-4160-b052-ee8d9c3cb001 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:36.592768 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9281a100-cd58-4160-b052-ee8d9c3cb001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:36.612193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281a100-cd58-4160-b052-ee8d9c3cb001 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:36.617723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:36.618661 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:36.618941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:36.619297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] PATH_INFO: `/v3/projects/be80b208dfae41ccb05e21e9d2a685a6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:36.620568 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be80b208dfae41ccb05e21e9d2a685a6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:36.622491 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:36.672832 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be80b208dfae41ccb05e21e9d2a685a6', 'request_id': 'req-627aa7be-8abe-45f1-9931-eca3b7b82a34'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.679608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be80b208dfae41ccb05e21e9d2a685a6', 'request_id': 'req-627aa7be-8abe-45f1-9931-eca3b7b82a34'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.687147 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be80b208dfae41ccb05e21e9d2a685a6', 'request_id': 'req-627aa7be-8abe-45f1-9931-eca3b7b82a34'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.699547 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] The token cache is being invalidate because project be80b208dfae41ccb05e21e9d2a685a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:10:36.699913 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.700786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.702673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1323/2645] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:10:36 2022] DELETE /identity/v3/projects/be80b208dfae41ccb05e21e9d2a685a6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:36.707246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfc639-16a0-4a9c-88bd-aefcacf8b2e3 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:36.725190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfc639-16a0-4a9c-88bd-aefcacf8b2e3 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:36.794575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:36.795453 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:36.795739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:36.796007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] PATH_INFO: `/v3/users/b1bde021bba94313b0fdce6ed387bbc1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:36.797231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1bde021bba94313b0fdce6ed387bbc1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:36.800712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:36.852633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1bde021bba94313b0fdce6ed387bbc1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.853299 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1bde021bba94313b0fdce6ed387bbc1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.853907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1bde021bba94313b0fdce6ed387bbc1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.861002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1bde021bba94313b0fdce6ed387bbc1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.876678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-666bfc1a-0689-43b2-81e0-531d684e3ff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1bde021bba94313b0fdce6ed387bbc1'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:36.890000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1323/2646] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:10:36 2022] DELETE /identity/v3/users/b1bde021bba94313b0fdce6ed387bbc1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:37.178306 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:37.180005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:37.200430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627aa7be-8abe-45f1-9931-eca3b7b82a34 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:37.229845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:37.264214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:37.265040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.265322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.265599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] PATH_INFO: `/v3/projects/a5352f05baa64793b2278682382acdf9` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.266761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5352f05baa64793b2278682382acdf9)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:37.268776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:37.314675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5352f05baa64793b2278682382acdf9', 'request_id': 'req-01bbe756-e54b-4cb8-b134-ca8d078615c2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:37.321785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5352f05baa64793b2278682382acdf9', 'request_id': 'req-01bbe756-e54b-4cb8-b134-ca8d078615c2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:37.329674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5352f05baa64793b2278682382acdf9', 'request_id': 'req-01bbe756-e54b-4cb8-b134-ca8d078615c2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:37.340373 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] The token cache is being invalidate because project a5352f05baa64793b2278682382acdf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:10:37.340666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:37.341297 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01bbe756-e54b-4cb8-b134-ca8d078615c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:37.342858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1324/2647] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:10:37 2022] DELETE /identity/v3/projects/a5352f05baa64793b2278682382acdf9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:10:37.349596 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce264e-5a04-49e2-a373-718091e5dae8 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.350060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce264e-5a04-49e2-a373-718091e5dae8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.350167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce264e-5a04-49e2-a373-718091e5dae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.368369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5bce264e-5a04-49e2-a373-718091e5dae8 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:37.369586 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5bce264e-5a04-49e2-a373-718091e5dae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:37.401471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:37.437236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1324/2648] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:37.446253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b853de-3fb8-4fc3-a2a2-a7c81b46d94d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.446544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b853de-3fb8-4fc3-a2a2-a7c81b46d94d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.446827 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b853de-3fb8-4fc3-a2a2-a7c81b46d94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.465398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-29b853de-3fb8-4fc3-a2a2-a7c81b46d94d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:37.466608 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-29b853de-3fb8-4fc3-a2a2-a7c81b46d94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:37.475092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1325/2649] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:37.479114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce264e-5a04-49e2-a373-718091e5dae8 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:37.488669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce264e-5a04-49e2-a373-718091e5dae8 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:37.494726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cb267-34f7-4477-bbd4-0a471247b036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:37.495381 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cb267-34f7-4477-bbd4-0a471247b036 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.495666 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cb267-34f7-4477-bbd4-0a471247b036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.495945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cb267-34f7-4477-bbd4-0a471247b036 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.496693 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cb267-34f7-4477-bbd4-0a471247b036 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:10:37.497003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cb267-34f7-4477-bbd4-0a471247b036 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:37.498587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cb267-34f7-4477-bbd4-0a471247b036 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:37.514637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1325/2650] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:10:37.515749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b853de-3fb8-4fc3-a2a2-a7c81b46d94d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:37.526162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b853de-3fb8-4fc3-a2a2-a7c81b46d94d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:37.530803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa66ef5-16d0-453e-827d-1e4bebf0fb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:37.531456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa66ef5-16d0-453e-827d-1e4bebf0fb47 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.531735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa66ef5-16d0-453e-827d-1e4bebf0fb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.532009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa66ef5-16d0-453e-827d-1e4bebf0fb47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.532734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa66ef5-16d0-453e-827d-1e4bebf0fb47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:37.534629 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa66ef5-16d0-453e-827d-1e4bebf0fb47 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:37.560593 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1326/2651] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:10:37 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:10:37.564925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cb267-34f7-4477-bbd4-0a471247b036 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:37.575081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cb267-34f7-4477-bbd4-0a471247b036 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:37.640893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ea8e3-6004-4d7d-bfa4-5121cbd1cfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:37.641954 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ea8e3-6004-4d7d-bfa4-5121cbd1cfbd admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.642398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ea8e3-6004-4d7d-bfa4-5121cbd1cfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.642823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ea8e3-6004-4d7d-bfa4-5121cbd1cfbd admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.643950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ea8e3-6004-4d7d-bfa4-5121cbd1cfbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:37.647877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ea8e3-6004-4d7d-bfa4-5121cbd1cfbd admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:37.671209 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1326/2652] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:10:37 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:37.675905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa66ef5-16d0-453e-827d-1e4bebf0fb47 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:37.686108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa66ef5-16d0-453e-827d-1e4bebf0fb47 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:37.690658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6147d-c26c-4f82-b508-b74fa81fd031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:37.691307 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6147d-c26c-4f82-b508-b74fa81fd031 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.691588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6147d-c26c-4f82-b508-b74fa81fd031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.691862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6147d-c26c-4f82-b508-b74fa81fd031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.692657 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6147d-c26c-4f82-b508-b74fa81fd031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:37.694239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6147d-c26c-4f82-b508-b74fa81fd031 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:37.704407 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1327/2653] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:37.708907 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ea8e3-6004-4d7d-bfa4-5121cbd1cfbd admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:37.718575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ea8e3-6004-4d7d-bfa4-5121cbd1cfbd admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:37.724372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cabe0-1bcd-4140-ae82-81ce43e0adae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:37.724657 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cabe0-1bcd-4140-ae82-81ce43e0adae admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.724941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cabe0-1bcd-4140-ae82-81ce43e0adae admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.725234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cabe0-1bcd-4140-ae82-81ce43e0adae admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.725937 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cabe0-1bcd-4140-ae82-81ce43e0adae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:37.727503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cabe0-1bcd-4140-ae82-81ce43e0adae admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:37.742769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1327/2654] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:37.747009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6147d-c26c-4f82-b508-b74fa81fd031 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:37.757826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6147d-c26c-4f82-b508-b74fa81fd031 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:37.762581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7a58d-4ae3-4825-9d95-58a19ee6007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:37.763454 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7a58d-4ae3-4825-9d95-58a19ee6007f admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.763736 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7a58d-4ae3-4825-9d95-58a19ee6007f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.764011 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7a58d-4ae3-4825-9d95-58a19ee6007f admin admin] PATH_INFO: `/v3/projects/34b6443b5e674042b9a8fcd2162d08b1/users/619148586cd6442496dbe8763cc169e8/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.779533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7a58d-4ae3-4825-9d95-58a19ee6007f admin admin] RBAC: Authorizing `identity:create_grant(project_id=34b6443b5e674042b9a8fcd2162d08b1, user_id=619148586cd6442496dbe8763cc169e8, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:37.782290 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7a58d-4ae3-4825-9d95-58a19ee6007f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:37.795355 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1328/2655] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:10:37 2022] PUT /identity/v3/projects/34b6443b5e674042b9a8fcd2162d08b1/users/619148586cd6442496dbe8763cc169e8/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:10:37.807650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63465897-f4cf-4f44-b205-32eff1dbbe97 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.807945 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63465897-f4cf-4f44-b205-32eff1dbbe97 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.808230 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63465897-f4cf-4f44-b205-32eff1dbbe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.835599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-63465897-f4cf-4f44-b205-32eff1dbbe97 None None] MFA Rules not processed for user `619148586cd6442496dbe8763cc169e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:37.836886 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-63465897-f4cf-4f44-b205-32eff1dbbe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:37.906129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1328/2656] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:37.917794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7a58d-4ae3-4825-9d95-58a19ee6007f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:37.919484 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0ad7a58d-4ae3-4825-9d95-58a19ee6007f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:37.978914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7a58d-4ae3-4825-9d95-58a19ee6007f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:37.982575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025cc5f-5fc0-4ceb-ab2d-818fadd80216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:37.983226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025cc5f-5fc0-4ceb-ab2d-818fadd80216 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:37.983507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025cc5f-5fc0-4ceb-ab2d-818fadd80216 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:37.983779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025cc5f-5fc0-4ceb-ab2d-818fadd80216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:37.991226 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:37.994019 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025cc5f-5fc0-4ceb-ab2d-818fadd80216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:37.996105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025cc5f-5fc0-4ceb-ab2d-818fadd80216 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:38.007293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1329/2657] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:47.902868 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:48.007643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:58.613668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63465897-f4cf-4f44-b205-32eff1dbbe97 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:58.675632 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63465897-f4cf-4f44-b205-32eff1dbbe97 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:58.680202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:58.681101 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:58.681387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:58.681674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] PATH_INFO: `/v3/users/619148586cd6442496dbe8763cc169e8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:58.682870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=619148586cd6442496dbe8763cc169e8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:58.684841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:58.729705 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '619148586cd6442496dbe8763cc169e8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:58.730348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '619148586cd6442496dbe8763cc169e8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:58.730947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '619148586cd6442496dbe8763cc169e8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:58.737911 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '619148586cd6442496dbe8763cc169e8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:58.749678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '619148586cd6442496dbe8763cc169e8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:58.762377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1329/2658] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:10:58 2022] DELETE /identity/v3/users/619148586cd6442496dbe8763cc169e8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:58.767031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e9a4c-274c-4d2e-9eea-76e6b1a05c6f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:58.767486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e9a4c-274c-4d2e-9eea-76e6b1a05c6f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:58.767571 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e9a4c-274c-4d2e-9eea-76e6b1a05c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:58.785493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-144e9a4c-274c-4d2e-9eea-76e6b1a05c6f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:58.786671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-144e9a4c-274c-4d2e-9eea-76e6b1a05c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:58.847416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1330/2659] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:58.857200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:58.858590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:58.930151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb116dd-bea5-43da-8f3d-a223951a84cc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:58.934334 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66fc0e-1ae1-4f2d-8128-3eb10c21e72c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:58.935087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66fc0e-1ae1-4f2d-8128-3eb10c21e72c service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:58.935542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66fc0e-1ae1-4f2d-8128-3eb10c21e72c service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:58.935887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66fc0e-1ae1-4f2d-8128-3eb10c21e72c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:58.946793 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66fc0e-1ae1-4f2d-8128-3eb10c21e72c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:58.948236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66fc0e-1ae1-4f2d-8128-3eb10c21e72c service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:58.958412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1330/2660] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:10:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:10:59.237284 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e9a4c-274c-4d2e-9eea-76e6b1a05c6f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:59.239276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-144e9a4c-274c-4d2e-9eea-76e6b1a05c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:59.251051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e9a4c-274c-4d2e-9eea-76e6b1a05c6f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:59.256881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:59.257690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.257972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.258250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] PATH_INFO: `/v3/projects/34b6443b5e674042b9a8fcd2162d08b1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.259417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34b6443b5e674042b9a8fcd2162d08b1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:59.261354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:59.314572 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34b6443b5e674042b9a8fcd2162d08b1', 'request_id': 'req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:59.321263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34b6443b5e674042b9a8fcd2162d08b1', 'request_id': 'req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:59.328545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34b6443b5e674042b9a8fcd2162d08b1', 'request_id': 'req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:59.339135 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] The token cache is being invalidate because project 34b6443b5e674042b9a8fcd2162d08b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:10:59.339418 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:59.340036 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dfb2fc5-c547-4fee-ac0b-736ce1795e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:10:59.341601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1331/2661] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:10:59 2022] DELETE /identity/v3/projects/34b6443b5e674042b9a8fcd2162d08b1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:10:59.346617 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803af78b-6403-4720-8c93-4baa484914d7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.346912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803af78b-6403-4720-8c93-4baa484914d7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.347247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803af78b-6403-4720-8c93-4baa484914d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.365419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-803af78b-6403-4720-8c93-4baa484914d7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:59.366640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-803af78b-6403-4720-8c93-4baa484914d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:59.384284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:59.427695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1331/2662] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:59.435519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4697a-7a44-4cd1-bd7b-4ce6523966a7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.435852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4697a-7a44-4cd1-bd7b-4ce6523966a7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.436133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4697a-7a44-4cd1-bd7b-4ce6523966a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.454650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-96b4697a-7a44-4cd1-bd7b-4ce6523966a7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:59.455837 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-96b4697a-7a44-4cd1-bd7b-4ce6523966a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:59.463574 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1332/2663] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:59.467580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803af78b-6403-4720-8c93-4baa484914d7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:59.477057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803af78b-6403-4720-8c93-4baa484914d7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:59.481565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfbe7c-a9df-41f6-a52b-8c2b4c993e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:59.482233 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfbe7c-a9df-41f6-a52b-8c2b4c993e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.482516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfbe7c-a9df-41f6-a52b-8c2b4c993e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.482787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfbe7c-a9df-41f6-a52b-8c2b4c993e92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.483510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfbe7c-a9df-41f6-a52b-8c2b4c993e92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:10:59.483813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfbe7c-a9df-41f6-a52b-8c2b4c993e92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:59.485421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfbe7c-a9df-41f6-a52b-8c2b4c993e92 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:59.497277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1332/2664] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:10:59.501446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4697a-7a44-4cd1-bd7b-4ce6523966a7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:59.510962 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4697a-7a44-4cd1-bd7b-4ce6523966a7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:59.515496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67895773-64d0-4e8f-b1a9-92d4adacf988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:59.516141 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895773-64d0-4e8f-b1a9-92d4adacf988 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.516458 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895773-64d0-4e8f-b1a9-92d4adacf988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.516738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895773-64d0-4e8f-b1a9-92d4adacf988 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.517425 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67895773-64d0-4e8f-b1a9-92d4adacf988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:59.519366 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67895773-64d0-4e8f-b1a9-92d4adacf988 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:59.544798 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1333/2665] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:10:59 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:59.549289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfbe7c-a9df-41f6-a52b-8c2b4c993e92 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:59.559566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfbe7c-a9df-41f6-a52b-8c2b4c993e92 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:59.615985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6df47-4e57-48b9-8039-0e38a4d8e553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:59.616719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6df47-4e57-48b9-8039-0e38a4d8e553 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.617005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6df47-4e57-48b9-8039-0e38a4d8e553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.617284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6df47-4e57-48b9-8039-0e38a4d8e553 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.618043 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6df47-4e57-48b9-8039-0e38a4d8e553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:59.619976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6df47-4e57-48b9-8039-0e38a4d8e553 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:59.639956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1333/2666] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:10:59 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:10:59.644136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67895773-64d0-4e8f-b1a9-92d4adacf988 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:59.654450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67895773-64d0-4e8f-b1a9-92d4adacf988 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:59.659048 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821844e-9027-4e69-a59b-2ad16778f185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:59.660236 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821844e-9027-4e69-a59b-2ad16778f185 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.660647 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821844e-9027-4e69-a59b-2ad16778f185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.660916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821844e-9027-4e69-a59b-2ad16778f185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.661673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821844e-9027-4e69-a59b-2ad16778f185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:59.663263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821844e-9027-4e69-a59b-2ad16778f185 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:59.674646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1334/2667] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:59.679372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6df47-4e57-48b9-8039-0e38a4d8e553 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:59.689704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6df47-4e57-48b9-8039-0e38a4d8e553 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:59.694318 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfa8c2-787e-421e-bf51-9334517ec09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:59.694977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfa8c2-787e-421e-bf51-9334517ec09b admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.695265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfa8c2-787e-421e-bf51-9334517ec09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.695542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfa8c2-787e-421e-bf51-9334517ec09b admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.696273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfa8c2-787e-421e-bf51-9334517ec09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:59.697888 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfa8c2-787e-421e-bf51-9334517ec09b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:59.707964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1334/2668] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:10:59.712095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821844e-9027-4e69-a59b-2ad16778f185 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:59.721735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821844e-9027-4e69-a59b-2ad16778f185 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:59.726962 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729814b7-1d6b-4330-ba93-59ae3f10c861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:59.727826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729814b7-1d6b-4330-ba93-59ae3f10c861 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.728104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729814b7-1d6b-4330-ba93-59ae3f10c861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.728411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729814b7-1d6b-4330-ba93-59ae3f10c861 admin admin] PATH_INFO: `/v3/projects/ea972be4a5774b17857d1483e5711840/users/4b27e68ff8f64c1abcd8a3ac08289ee1/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.743070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729814b7-1d6b-4330-ba93-59ae3f10c861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea972be4a5774b17857d1483e5711840, user_id=4b27e68ff8f64c1abcd8a3ac08289ee1, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:59.745753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729814b7-1d6b-4330-ba93-59ae3f10c861 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:59.759090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1335/2669] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:10:59 2022] PUT /identity/v3/projects/ea972be4a5774b17857d1483e5711840/users/4b27e68ff8f64c1abcd8a3ac08289ee1/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:10:59.770281 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cc716-988d-444f-a323-6adb4ebc771a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.770503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cc716-988d-444f-a323-6adb4ebc771a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.770788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cc716-988d-444f-a323-6adb4ebc771a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.798165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-637cc716-988d-444f-a323-6adb4ebc771a None None] MFA Rules not processed for user `4b27e68ff8f64c1abcd8a3ac08289ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:10:59.799613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-637cc716-988d-444f-a323-6adb4ebc771a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:10:59.862478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1335/2670] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:10:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:10:59.867603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729814b7-1d6b-4330-ba93-59ae3f10c861 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:59.888730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:10:59.932223 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729814b7-1d6b-4330-ba93-59ae3f10c861 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:59.937111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b490cd-48d8-4bbd-93c5-c16e7b8e6a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:10:59.937770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b490cd-48d8-4bbd-93c5-c16e7b8e6a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:10:59.938055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b490cd-48d8-4bbd-93c5-c16e7b8e6a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:10:59.938447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b490cd-48d8-4bbd-93c5-c16e7b8e6a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:10:59.939016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b490cd-48d8-4bbd-93c5-c16e7b8e6a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:10:59.941032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b490cd-48d8-4bbd-93c5-c16e7b8e6a2b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:10:59.971845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1336/2671] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:10:59 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:10:59.973894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cc716-988d-444f-a323-6adb4ebc771a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:10:59.984340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cc716-988d-444f-a323-6adb4ebc771a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:10:59.996499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:00.041463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb5395-271b-473c-84b1-5d373d6c23ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.042142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5395-271b-473c-84b1-5d373d6c23ac admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.042420 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5395-271b-473c-84b1-5d373d6c23ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.042694 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5395-271b-473c-84b1-5d373d6c23ac admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.043420 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb5395-271b-473c-84b1-5d373d6c23ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.045345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb5395-271b-473c-84b1-5d373d6c23ac admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.069316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1336/2672] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:10:59 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:11:00.069991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b490cd-48d8-4bbd-93c5-c16e7b8e6a2b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:00.080740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b490cd-48d8-4bbd-93c5-c16e7b8e6a2b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:00.086186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9de46-c8c4-472c-b0a2-08aada40425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.086844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9de46-c8c4-472c-b0a2-08aada40425c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.087123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9de46-c8c4-472c-b0a2-08aada40425c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.087397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9de46-c8c4-472c-b0a2-08aada40425c admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.088132 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9de46-c8c4-472c-b0a2-08aada40425c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.089752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9de46-c8c4-472c-b0a2-08aada40425c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.101996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1337/2673] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 18:11:00.105921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb5395-271b-473c-84b1-5d373d6c23ac admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:00.116930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb5395-271b-473c-84b1-5d373d6c23ac admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:00.121985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a9bc8-6730-4043-951a-fc74256cbcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.122647 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a9bc8-6730-4043-951a-fc74256cbcdc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.123007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a9bc8-6730-4043-951a-fc74256cbcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.123305 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a9bc8-6730-4043-951a-fc74256cbcdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.124130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a9bc8-6730-4043-951a-fc74256cbcdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.125870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a9bc8-6730-4043-951a-fc74256cbcdc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.136559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1337/2674] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:11:00.140872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9de46-c8c4-472c-b0a2-08aada40425c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:00.151623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9de46-c8c4-472c-b0a2-08aada40425c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:00.156879 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b2c7b-68d1-4b5a-887c-73b5c588faa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.157898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b2c7b-68d1-4b5a-887c-73b5c588faa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.158293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b2c7b-68d1-4b5a-887c-73b5c588faa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.158559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b2c7b-68d1-4b5a-887c-73b5c588faa6 admin admin] PATH_INFO: `/v3/projects/df9f384d2af842cc822caadd072ac22c/users/dc289e169ef343bb8498fce505b64404/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.174253 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b2c7b-68d1-4b5a-887c-73b5c588faa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df9f384d2af842cc822caadd072ac22c, user_id=dc289e169ef343bb8498fce505b64404, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.177336 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b2c7b-68d1-4b5a-887c-73b5c588faa6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.189595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1338/2675] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:11:00 2022] PUT /identity/v3/projects/df9f384d2af842cc822caadd072ac22c/users/dc289e169ef343bb8498fce505b64404/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:00.199974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da11f0a-02af-4076-8a19-a09719e7634e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.200265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da11f0a-02af-4076-8a19-a09719e7634e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.200581 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da11f0a-02af-4076-8a19-a09719e7634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.227587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8da11f0a-02af-4076-8a19-a09719e7634e None None] MFA Rules not processed for user `dc289e169ef343bb8498fce505b64404`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:00.228813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8da11f0a-02af-4076-8a19-a09719e7634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:00.292640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1338/2676] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:00.297108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b2c7b-68d1-4b5a-887c-73b5c588faa6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:00.359730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b2c7b-68d1-4b5a-887c-73b5c588faa6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:00.364806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a6d5b-2785-4778-9a7e-1503d711c60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.365540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a6d5b-2785-4778-9a7e-1503d711c60b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.365859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a6d5b-2785-4778-9a7e-1503d711c60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.366155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a6d5b-2785-4778-9a7e-1503d711c60b admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.366851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a6d5b-2785-4778-9a7e-1503d711c60b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.368775 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a6d5b-2785-4778-9a7e-1503d711c60b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.396076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1339/2677] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:11:00 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:11:00.400111 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da11f0a-02af-4076-8a19-a09719e7634e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:00.411321 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da11f0a-02af-4076-8a19-a09719e7634e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:00.468685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59a375-4b8b-4d84-a59f-044adf83b012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.469350 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59a375-4b8b-4d84-a59f-044adf83b012 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.469629 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59a375-4b8b-4d84-a59f-044adf83b012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.469901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59a375-4b8b-4d84-a59f-044adf83b012 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.470579 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59a375-4b8b-4d84-a59f-044adf83b012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.472521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59a375-4b8b-4d84-a59f-044adf83b012 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.494404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1339/2678] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:11:00 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:11:00.495680 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a6d5b-2785-4778-9a7e-1503d711c60b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:00.506596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a6d5b-2785-4778-9a7e-1503d711c60b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:00.511187 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95af7db-559f-40b3-9b73-093ef0e05f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.511838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95af7db-559f-40b3-9b73-093ef0e05f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.512158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95af7db-559f-40b3-9b73-093ef0e05f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.512479 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95af7db-559f-40b3-9b73-093ef0e05f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.513209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95af7db-559f-40b3-9b73-093ef0e05f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.514808 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95af7db-559f-40b3-9b73-093ef0e05f8a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.525944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1340/2679] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:11:00.529910 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59a375-4b8b-4d84-a59f-044adf83b012 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:00.540096 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59a375-4b8b-4d84-a59f-044adf83b012 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:00.544729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a9cc-c0e8-48e6-a5b4-f705923cb601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.545542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a9cc-c0e8-48e6-a5b4-f705923cb601 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.545822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a9cc-c0e8-48e6-a5b4-f705923cb601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.546095 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a9cc-c0e8-48e6-a5b4-f705923cb601 admin admin] PATH_INFO: `/v3/projects/6d267257b9f546fe9a89773480f69842/users/a39ac7c8e5af4149a9289d7aa887c8e5/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.558035 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:00.561190 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a9cc-c0e8-48e6-a5b4-f705923cb601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d267257b9f546fe9a89773480f69842, user_id=a39ac7c8e5af4149a9289d7aa887c8e5, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.564208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a9cc-c0e8-48e6-a5b4-f705923cb601 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.577340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1340/2680] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:11:00 2022] PUT /identity/v3/projects/6d267257b9f546fe9a89773480f69842/users/a39ac7c8e5af4149a9289d7aa887c8e5/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:00.581251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95af7db-559f-40b3-9b73-093ef0e05f8a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:00.591863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95af7db-559f-40b3-9b73-093ef0e05f8a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:00.633361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:00.649209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69535285-4170-409b-a1fb-6ade81364c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.649888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69535285-4170-409b-a1fb-6ade81364c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.650175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69535285-4170-409b-a1fb-6ade81364c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.650456 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69535285-4170-409b-a1fb-6ade81364c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.651186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69535285-4170-409b-a1fb-6ade81364c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.652859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69535285-4170-409b-a1fb-6ade81364c83 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.662586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1341/2681] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:11:00.666635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a9cc-c0e8-48e6-a5b4-f705923cb601 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:00.677573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a9cc-c0e8-48e6-a5b4-f705923cb601 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:00.682178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7051c04-0260-40a7-aaac-b1fda8982dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.683057 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7051c04-0260-40a7-aaac-b1fda8982dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.683343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7051c04-0260-40a7-aaac-b1fda8982dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.683624 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7051c04-0260-40a7-aaac-b1fda8982dc6 admin admin] PATH_INFO: `/v3/projects/6d267257b9f546fe9a89773480f69842/users/a39ac7c8e5af4149a9289d7aa887c8e5/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.685642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7051c04-0260-40a7-aaac-b1fda8982dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d267257b9f546fe9a89773480f69842, user_id=a39ac7c8e5af4149a9289d7aa887c8e5, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.688274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7051c04-0260-40a7-aaac-b1fda8982dc6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.699093 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1341/2682] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:11:00 2022] PUT /identity/v3/projects/6d267257b9f546fe9a89773480f69842/users/a39ac7c8e5af4149a9289d7aa887c8e5/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:00.709281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521b7a8-77a4-4de7-8a8b-224b74565a4a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.709578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521b7a8-77a4-4de7-8a8b-224b74565a4a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.709861 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521b7a8-77a4-4de7-8a8b-224b74565a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.735606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1521b7a8-77a4-4de7-8a8b-224b74565a4a None None] MFA Rules not processed for user `a39ac7c8e5af4149a9289d7aa887c8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:00.736850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1521b7a8-77a4-4de7-8a8b-224b74565a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:00.805653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1342/2683] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:00.817403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7051c04-0260-40a7-aaac-b1fda8982dc6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:00.819165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a7051c04-0260-40a7-aaac-b1fda8982dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:00.877024 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7051c04-0260-40a7-aaac-b1fda8982dc6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:00.881427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57652f5b-76e5-4556-a79d-451e2ce1a185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:00.882074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57652f5b-76e5-4556-a79d-451e2ce1a185 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:00.882359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57652f5b-76e5-4556-a79d-451e2ce1a185 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:00.882637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57652f5b-76e5-4556-a79d-451e2ce1a185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:00.939471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57652f5b-76e5-4556-a79d-451e2ce1a185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:00.941609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57652f5b-76e5-4556-a79d-451e2ce1a185 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:00.950641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1342/2684] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:11:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:11:02.340451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521b7a8-77a4-4de7-8a8b-224b74565a4a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:02.342027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1521b7a8-77a4-4de7-8a8b-224b74565a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:02.416848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521b7a8-77a4-4de7-8a8b-224b74565a4a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:02.421514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:02.422470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:02.422761 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:02.423043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] PATH_INFO: `/v3/users/0b1ca26e28cf400ab886665c3f4f6e4e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:02.424525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b1ca26e28cf400ab886665c3f4f6e4e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:02.427204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:02.457787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57652f5b-76e5-4556-a79d-451e2ce1a185 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:02.459401 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-57652f5b-76e5-4556-a79d-451e2ce1a185 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:02.471081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57652f5b-76e5-4556-a79d-451e2ce1a185 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:02.474921 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495349dc-7ea4-40dc-b8cd-1a40d22db183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:02.475636 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495349dc-7ea4-40dc-b8cd-1a40d22db183 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:02.475992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495349dc-7ea4-40dc-b8cd-1a40d22db183 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:02.476382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495349dc-7ea4-40dc-b8cd-1a40d22db183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:02.476706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b1ca26e28cf400ab886665c3f4f6e4e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:02.483948 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b1ca26e28cf400ab886665c3f4f6e4e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:02.484695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b1ca26e28cf400ab886665c3f4f6e4e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:02.493204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:02.498272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b1ca26e28cf400ab886665c3f4f6e4e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:02.506737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55c6bec5-7456-45c2-b726-d1953657a267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b1ca26e28cf400ab886665c3f4f6e4e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:02.512782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1343/2685] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:11:02 2022] DELETE /identity/v3/users/0b1ca26e28cf400ab886665c3f4f6e4e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:02.517125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fff10-3fcb-4061-9bf2-36fa40e2a6ab None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:02.517423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fff10-3fcb-4061-9bf2-36fa40e2a6ab None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:02.517738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fff10-3fcb-4061-9bf2-36fa40e2a6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:02.536820 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-199fff10-3fcb-4061-9bf2-36fa40e2a6ab None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:02.537964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-199fff10-3fcb-4061-9bf2-36fa40e2a6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:02.540770 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495349dc-7ea4-40dc-b8cd-1a40d22db183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:02.542202 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495349dc-7ea4-40dc-b8cd-1a40d22db183 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:02.552109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1343/2686] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:11:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:11:02.599029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1344/2687] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:11:02.608672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495349dc-7ea4-40dc-b8cd-1a40d22db183 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:02.610303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-495349dc-7ea4-40dc-b8cd-1a40d22db183 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:02.649025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:02.676359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495349dc-7ea4-40dc-b8cd-1a40d22db183 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:02.728431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bada3c-914b-4df8-8559-d72285ef8a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:02.729090 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bada3c-914b-4df8-8559-d72285ef8a81 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:02.729373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bada3c-914b-4df8-8559-d72285ef8a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:02.729649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bada3c-914b-4df8-8559-d72285ef8a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:02.740036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bada3c-914b-4df8-8559-d72285ef8a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:02.741496 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bada3c-914b-4df8-8559-d72285ef8a81 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:02.751337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1344/2688] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:11:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:11:03.054218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fff10-3fcb-4061-9bf2-36fa40e2a6ab None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:03.055515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-199fff10-3fcb-4061-9bf2-36fa40e2a6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:03.077704 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fff10-3fcb-4061-9bf2-36fa40e2a6ab None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:03.088191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:03.089181 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.089528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.089867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] PATH_INFO: `/v3/projects/24c58716f06042efbcc3c9b4da0fb07d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.091183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c58716f06042efbcc3c9b4da0fb07d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:03.093195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:03.137201 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bada3c-914b-4df8-8559-d72285ef8a81 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:03.138857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-40bada3c-914b-4df8-8559-d72285ef8a81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:03.158613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c58716f06042efbcc3c9b4da0fb07d', 'request_id': 'req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:03.165804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c58716f06042efbcc3c9b4da0fb07d', 'request_id': 'req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:03.173585 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c58716f06042efbcc3c9b4da0fb07d', 'request_id': 'req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:03.187377 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] The token cache is being invalidate because project 24c58716f06042efbcc3c9b4da0fb07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:11:03.187673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:03.188371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bbfc5613-cf8e-424c-b05f-bf50b1bc8624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:03.189934 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1345/2689] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:11:03 2022] DELETE /identity/v3/projects/24c58716f06042efbcc3c9b4da0fb07d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:03.195462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40238b-7b96-4b6d-9566-96796a3ed31d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.195751 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40238b-7b96-4b6d-9566-96796a3ed31d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.196029 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40238b-7b96-4b6d-9566-96796a3ed31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.200504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bada3c-914b-4df8-8559-d72285ef8a81 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:03.216864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5b40238b-7b96-4b6d-9566-96796a3ed31d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:03.218071 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5b40238b-7b96-4b6d-9566-96796a3ed31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:03.257219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c268fd-f7d3-4775-be3f-5e6b2e75f60b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:03.257883 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c268fd-f7d3-4775-be3f-5e6b2e75f60b service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.258170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c268fd-f7d3-4775-be3f-5e6b2e75f60b service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.258480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c268fd-f7d3-4775-be3f-5e6b2e75f60b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.260255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-15c268fd-f7d3-4775-be3f-5e6b2e75f60b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:03.285420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1346/2690] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:11:03.293283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19454c-a54f-4d5d-8c29-8719268b9b88 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.293598 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19454c-a54f-4d5d-8c29-8719268b9b88 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.293903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19454c-a54f-4d5d-8c29-8719268b9b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.311928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8a19454c-a54f-4d5d-8c29-8719268b9b88 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:03.313172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8a19454c-a54f-4d5d-8c29-8719268b9b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:03.322161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:03.322842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1347/2691] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:11:03.324889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c268fd-f7d3-4775-be3f-5e6b2e75f60b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:03.326834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19454c-a54f-4d5d-8c29-8719268b9b88 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:03.327144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c268fd-f7d3-4775-be3f-5e6b2e75f60b service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:03.331083 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:03.338646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1345/2692] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:11:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:11:03.339168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19454c-a54f-4d5d-8c29-8719268b9b88 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:03.345570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ba7d9-ad66-4bc8-b324-40474d06ba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:03.346228 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ba7d9-ad66-4bc8-b324-40474d06ba7a admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.346519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ba7d9-ad66-4bc8-b324-40474d06ba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.347007 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ba7d9-ad66-4bc8-b324-40474d06ba7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.347749 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ba7d9-ad66-4bc8-b324-40474d06ba7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:11:03.348062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ba7d9-ad66-4bc8-b324-40474d06ba7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:03.349729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ba7d9-ad66-4bc8-b324-40474d06ba7a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:03.361625 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1348/2693] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 18:11:03.366335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c268fd-f7d3-4775-be3f-5e6b2e75f60b service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:03.377231 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c268fd-f7d3-4775-be3f-5e6b2e75f60b service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:03.382877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05b180-e7f9-44b3-8af2-bb05bbc72cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:03.383614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05b180-e7f9-44b3-8af2-bb05bbc72cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.383894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05b180-e7f9-44b3-8af2-bb05bbc72cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.384223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05b180-e7f9-44b3-8af2-bb05bbc72cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.384971 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05b180-e7f9-44b3-8af2-bb05bbc72cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:03.387553 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05b180-e7f9-44b3-8af2-bb05bbc72cf8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:03.413643 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1346/2694] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:11:03 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:11:03.418050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ba7d9-ad66-4bc8-b324-40474d06ba7a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:03.428881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ba7d9-ad66-4bc8-b324-40474d06ba7a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:03.485752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e97da-d1fa-4ec7-a9fa-7ce8780cc57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:03.486523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e97da-d1fa-4ec7-a9fa-7ce8780cc57a admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.486809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e97da-d1fa-4ec7-a9fa-7ce8780cc57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.487105 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e97da-d1fa-4ec7-a9fa-7ce8780cc57a admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.487855 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e97da-d1fa-4ec7-a9fa-7ce8780cc57a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:03.489800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e97da-d1fa-4ec7-a9fa-7ce8780cc57a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:03.509575 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1349/2695] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:11:03 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:11:03.513739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05b180-e7f9-44b3-8af2-bb05bbc72cf8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:03.524447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05b180-e7f9-44b3-8af2-bb05bbc72cf8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:03.528923 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245b727-ed85-4ff9-818c-94f722df0f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:03.529575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245b727-ed85-4ff9-818c-94f722df0f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.529856 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245b727-ed85-4ff9-818c-94f722df0f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.530131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245b727-ed85-4ff9-818c-94f722df0f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.530851 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245b727-ed85-4ff9-818c-94f722df0f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:03.532471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245b727-ed85-4ff9-818c-94f722df0f21 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:03.543052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1347/2696] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:11:03.546902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e97da-d1fa-4ec7-a9fa-7ce8780cc57a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:03.557069 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e97da-d1fa-4ec7-a9fa-7ce8780cc57a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:03.561606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea7f2e-d9da-447d-9818-a24befd73ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:03.562258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7f2e-d9da-447d-9818-a24befd73ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.562544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7f2e-d9da-447d-9818-a24befd73ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.562826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7f2e-d9da-447d-9818-a24befd73ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.563551 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea7f2e-d9da-447d-9818-a24befd73ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:03.565233 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea7f2e-d9da-447d-9818-a24befd73ef1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:03.576130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1350/2697] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:11:03.580236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245b727-ed85-4ff9-818c-94f722df0f21 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:03.591574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245b727-ed85-4ff9-818c-94f722df0f21 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:03.596742 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7388d6-5d61-479b-958e-f01c652c1ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:03.597834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7388d6-5d61-479b-958e-f01c652c1ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.598126 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7388d6-5d61-479b-958e-f01c652c1ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.598905 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7388d6-5d61-479b-958e-f01c652c1ef8 admin admin] PATH_INFO: `/v3/projects/f7cc308152b04d2db04bd8ec2da31dec/users/5dcf34766268492e819fbb971d394194/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.614564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7388d6-5d61-479b-958e-f01c652c1ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7cc308152b04d2db04bd8ec2da31dec, user_id=5dcf34766268492e819fbb971d394194, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:03.617244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7388d6-5d61-479b-958e-f01c652c1ef8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:03.630623 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1348/2698] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:11:03 2022] PUT /identity/v3/projects/f7cc308152b04d2db04bd8ec2da31dec/users/5dcf34766268492e819fbb971d394194/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:03.641655 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85f2e6-fbf5-4cc4-a3f2-8dbf0c708c85 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.641945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85f2e6-fbf5-4cc4-a3f2-8dbf0c708c85 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.642227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85f2e6-fbf5-4cc4-a3f2-8dbf0c708c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.669965 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2e85f2e6-fbf5-4cc4-a3f2-8dbf0c708c85 None None] MFA Rules not processed for user `5dcf34766268492e819fbb971d394194`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:03.671208 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2e85f2e6-fbf5-4cc4-a3f2-8dbf0c708c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:03.735488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1351/2699] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:03.747509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7388d6-5d61-479b-958e-f01c652c1ef8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:03.749267 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6b7388d6-5d61-479b-958e-f01c652c1ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:03.807789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7388d6-5d61-479b-958e-f01c652c1ef8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:03.811427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c53a2-819d-46a5-bbbb-f03e22c38651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:03.812067 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c53a2-819d-46a5-bbbb-f03e22c38651 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:03.812382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c53a2-819d-46a5-bbbb-f03e22c38651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:03.812661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c53a2-819d-46a5-bbbb-f03e22c38651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:03.821916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c53a2-819d-46a5-bbbb-f03e22c38651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:03.823346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c53a2-819d-46a5-bbbb-f03e22c38651 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:03.832583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1349/2700] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:11:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:11:09.267429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85f2e6-fbf5-4cc4-a3f2-8dbf0c708c85 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:09.268842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2e85f2e6-fbf5-4cc4-a3f2-8dbf0c708c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:09.329673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:09.335215 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85f2e6-fbf5-4cc4-a3f2-8dbf0c708c85 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:09.340062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:09.340964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:09.341467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:09.341753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] PATH_INFO: `/v3/users/4b27e68ff8f64c1abcd8a3ac08289ee1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:09.342953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b27e68ff8f64c1abcd8a3ac08289ee1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:09.344926 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:09.389271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b27e68ff8f64c1abcd8a3ac08289ee1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:09.397052 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b27e68ff8f64c1abcd8a3ac08289ee1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:09.397713 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b27e68ff8f64c1abcd8a3ac08289ee1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:09.409726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b27e68ff8f64c1abcd8a3ac08289ee1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:09.418769 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b27e68ff8f64c1abcd8a3ac08289ee1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:09.425326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1352/2701] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:11:09 2022] DELETE /identity/v3/users/4b27e68ff8f64c1abcd8a3ac08289ee1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:09.431372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5a072-6633-4509-b281-e7720161a559 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:09.431835 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5a072-6633-4509-b281-e7720161a559 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:09.432182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5a072-6633-4509-b281-e7720161a559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:09.454278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b8e5a072-6633-4509-b281-e7720161a559 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:09.455554 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b8e5a072-6633-4509-b281-e7720161a559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:09.515038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1350/2702] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:09.525490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:09.526878 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:09.587722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847f9af-6dd1-4414-96cf-c6e809066b61 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:09.591349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc454b2-fe95-430e-a3ef-ec83e29e25fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:09.591996 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc454b2-fe95-430e-a3ef-ec83e29e25fe service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:09.592276 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc454b2-fe95-430e-a3ef-ec83e29e25fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:09.592584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc454b2-fe95-430e-a3ef-ec83e29e25fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:09.602361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc454b2-fe95-430e-a3ef-ec83e29e25fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:09.603788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc454b2-fe95-430e-a3ef-ec83e29e25fe service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:09.614079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1353/2703] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:11:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:11:09.890039 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5a072-6633-4509-b281-e7720161a559 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:09.891683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b8e5a072-6633-4509-b281-e7720161a559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:09.904738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5a072-6633-4509-b281-e7720161a559 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:09.910276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:09.911968 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:09.912337 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:09.912620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] PATH_INFO: `/v3/projects/ea972be4a5774b17857d1483e5711840` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:09.914426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea972be4a5774b17857d1483e5711840)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:09.915898 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:09.955814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:09.966413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea972be4a5774b17857d1483e5711840', 'request_id': 'req-9b094cda-03a3-4c9f-8673-d0e804aad9a8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:09.973685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea972be4a5774b17857d1483e5711840', 'request_id': 'req-9b094cda-03a3-4c9f-8673-d0e804aad9a8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:09.979266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea972be4a5774b17857d1483e5711840', 'request_id': 'req-9b094cda-03a3-4c9f-8673-d0e804aad9a8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:09.990422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] The token cache is being invalidate because project ea972be4a5774b17857d1483e5711840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:11:09.990716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:09.991352 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:09.992928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1351/2704] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:11:09 2022] DELETE /identity/v3/projects/ea972be4a5774b17857d1483e5711840 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:09.996898 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc454b2-fe95-430e-a3ef-ec83e29e25fe service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:10.006862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc454b2-fe95-430e-a3ef-ec83e29e25fe service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:10.062149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:10.063035 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:10.063309 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:10.063580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] PATH_INFO: `/v3/users/dc289e169ef343bb8498fce505b64404` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:10.064758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc289e169ef343bb8498fce505b64404)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:10.066662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:10.088810 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:10.114961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc289e169ef343bb8498fce505b64404'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.121750 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc289e169ef343bb8498fce505b64404'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.122398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc289e169ef343bb8498fce505b64404'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.134460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc289e169ef343bb8498fce505b64404'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.141459 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc289e169ef343bb8498fce505b64404'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.147848 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1354/2705] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:11:09 2022] DELETE /identity/v3/users/dc289e169ef343bb8498fce505b64404 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:11:10.425914 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:10.427527 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:10.441182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b094cda-03a3-4c9f-8673-d0e804aad9a8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:10.500394 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:10.501270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:10.501559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:10.501850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] PATH_INFO: `/v3/projects/df9f384d2af842cc822caadd072ac22c` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:10.503029 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=df9f384d2af842cc822caadd072ac22c)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:10.505619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:10.554422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df9f384d2af842cc822caadd072ac22c', 'request_id': 'req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.561643 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df9f384d2af842cc822caadd072ac22c', 'request_id': 'req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.567288 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df9f384d2af842cc822caadd072ac22c', 'request_id': 'req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.579088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] The token cache is being invalidate because project df9f384d2af842cc822caadd072ac22c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:11:10.579374 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.580006 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.581583 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1352/2706] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:11:10 2022] DELETE /identity/v3/projects/df9f384d2af842cc822caadd072ac22c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:10.586436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:10.587840 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:10.601138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921ff03-22b8-4fc9-b859-67764e74bdce admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:10.656274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:10.657162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:10.657441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:10.657716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] PATH_INFO: `/v3/users/a39ac7c8e5af4149a9289d7aa887c8e5` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:10.658887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39ac7c8e5af4149a9289d7aa887c8e5)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:10.661496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:10.706945 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39ac7c8e5af4149a9289d7aa887c8e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.715125 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39ac7c8e5af4149a9289d7aa887c8e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.715813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39ac7c8e5af4149a9289d7aa887c8e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.728540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39ac7c8e5af4149a9289d7aa887c8e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.736001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3078fc56-2077-4f6f-9170-1f24610ec376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39ac7c8e5af4149a9289d7aa887c8e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:10.742519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1355/2707] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:11:10 2022] DELETE /identity/v3/users/a39ac7c8e5af4149a9289d7aa887c8e5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:11.017522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:11.019088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:11.034906 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a3c1f-d6e7-4320-8908-d42f3f44c65e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:11.090774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:11.099970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:11.101293 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.101695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.102154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] PATH_INFO: `/v3/projects/6d267257b9f546fe9a89773480f69842` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.103609 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d267257b9f546fe9a89773480f69842)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:11.105570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:11.151382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d267257b9f546fe9a89773480f69842', 'request_id': 'req-27ff5bb4-8492-477f-bf9d-9216b6213aff'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:11.158633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d267257b9f546fe9a89773480f69842', 'request_id': 'req-27ff5bb4-8492-477f-bf9d-9216b6213aff'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:11.164605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d267257b9f546fe9a89773480f69842', 'request_id': 'req-27ff5bb4-8492-477f-bf9d-9216b6213aff'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:11.175621 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] The token cache is being invalidate because project 6d267257b9f546fe9a89773480f69842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:11:11.175902 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:11.177389 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-27ff5bb4-8492-477f-bf9d-9216b6213aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:11.178909 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1353/2708] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:11:11 2022] DELETE /identity/v3/projects/6d267257b9f546fe9a89773480f69842 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:11.183863 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ee03-2ace-4554-b3fa-9c413204b2ed None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.185626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ee03-2ace-4554-b3fa-9c413204b2ed None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.185626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ee03-2ace-4554-b3fa-9c413204b2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.203009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a4a9ee03-2ace-4554-b3fa-9c413204b2ed None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:11.204221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a4a9ee03-2ace-4554-b3fa-9c413204b2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:11.217081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:11.265953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1356/2709] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:11.275637 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b66586-1731-4963-a3b6-3b9ad30d4245 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.275926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b66586-1731-4963-a3b6-3b9ad30d4245 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.276207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b66586-1731-4963-a3b6-3b9ad30d4245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.295428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c2b66586-1731-4963-a3b6-3b9ad30d4245 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:11.296672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c2b66586-1731-4963-a3b6-3b9ad30d4245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:11.307529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1354/2710] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:11.310530 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9ee03-2ace-4554-b3fa-9c413204b2ed None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:11.320646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9ee03-2ace-4554-b3fa-9c413204b2ed None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:11.325288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b666006-f843-457a-a62a-a98a86627561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:11.325940 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666006-f843-457a-a62a-a98a86627561 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.326224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666006-f843-457a-a62a-a98a86627561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.326504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666006-f843-457a-a62a-a98a86627561 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.327230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b666006-f843-457a-a62a-a98a86627561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:11:11.327541 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b666006-f843-457a-a62a-a98a86627561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:11.329175 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b666006-f843-457a-a62a-a98a86627561 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:11.341346 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1357/2711] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:11:11.345407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b66586-1731-4963-a3b6-3b9ad30d4245 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:11.355412 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b66586-1731-4963-a3b6-3b9ad30d4245 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:11.360074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0c5d8-a8a7-4aec-914c-398685046f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:11.360760 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0c5d8-a8a7-4aec-914c-398685046f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.361046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0c5d8-a8a7-4aec-914c-398685046f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.361325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0c5d8-a8a7-4aec-914c-398685046f34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.362028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0c5d8-a8a7-4aec-914c-398685046f34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:11.364061 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0c5d8-a8a7-4aec-914c-398685046f34 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:11.388788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1355/2712] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:11:11 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:11:11.392802 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b666006-f843-457a-a62a-a98a86627561 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:11.403199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b666006-f843-457a-a62a-a98a86627561 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:11.459525 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd7f0b-eafc-4db0-a127-204f525b1ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:11.460183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd7f0b-eafc-4db0-a127-204f525b1ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.460493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd7f0b-eafc-4db0-a127-204f525b1ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.460771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd7f0b-eafc-4db0-a127-204f525b1ec1 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.461452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd7f0b-eafc-4db0-a127-204f525b1ec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:11.463339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd7f0b-eafc-4db0-a127-204f525b1ec1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:11.486170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1358/2713] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:11:11 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:11:11.490366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0c5d8-a8a7-4aec-914c-398685046f34 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:11.500701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0c5d8-a8a7-4aec-914c-398685046f34 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:11.505967 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0721d6c-ed00-4aff-8b81-b9c738d3a0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:11.506614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0721d6c-ed00-4aff-8b81-b9c738d3a0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.506893 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0721d6c-ed00-4aff-8b81-b9c738d3a0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.507166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0721d6c-ed00-4aff-8b81-b9c738d3a0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.507884 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0721d6c-ed00-4aff-8b81-b9c738d3a0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:11.509556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0721d6c-ed00-4aff-8b81-b9c738d3a0f6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:11.519999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1356/2714] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:11:11.525032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd7f0b-eafc-4db0-a127-204f525b1ec1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:11.535171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd7f0b-eafc-4db0-a127-204f525b1ec1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:11.540446 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1900c-6548-4aaa-93a4-1174d4f02bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:11.541103 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1900c-6548-4aaa-93a4-1174d4f02bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.541382 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1900c-6548-4aaa-93a4-1174d4f02bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.541681 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1900c-6548-4aaa-93a4-1174d4f02bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.542405 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1900c-6548-4aaa-93a4-1174d4f02bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:11.544001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1900c-6548-4aaa-93a4-1174d4f02bb5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:11.554792 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1359/2715] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:11:11.558719 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0721d6c-ed00-4aff-8b81-b9c738d3a0f6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:11.569295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0721d6c-ed00-4aff-8b81-b9c738d3a0f6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:11.573849 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987f4ba-78ec-4cf7-9b82-859a81f01726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:11.574722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987f4ba-78ec-4cf7-9b82-859a81f01726 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.575001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987f4ba-78ec-4cf7-9b82-859a81f01726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.575274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987f4ba-78ec-4cf7-9b82-859a81f01726 admin admin] PATH_INFO: `/v3/projects/7a1469ed86624fbe90592a6ded1740ad/users/1f2ac51306654b26ad063a6f559f3bfb/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.590238 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987f4ba-78ec-4cf7-9b82-859a81f01726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1469ed86624fbe90592a6ded1740ad, user_id=1f2ac51306654b26ad063a6f559f3bfb, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:11.592920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987f4ba-78ec-4cf7-9b82-859a81f01726 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:11.606030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1357/2716] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:11:11 2022] PUT /identity/v3/projects/7a1469ed86624fbe90592a6ded1740ad/users/1f2ac51306654b26ad063a6f559f3bfb/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:11.611234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65679e36-00b1-434b-a04e-85ab839e85a1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.611671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65679e36-00b1-434b-a04e-85ab839e85a1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.611797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65679e36-00b1-434b-a04e-85ab839e85a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.630243 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-65679e36-00b1-434b-a04e-85ab839e85a1 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:11.631438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-65679e36-00b1-434b-a04e-85ab839e85a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:11.689782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1360/2717] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:11.700486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987f4ba-78ec-4cf7-9b82-859a81f01726 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:11.702349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6987f4ba-78ec-4cf7-9b82-859a81f01726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:11.758682 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987f4ba-78ec-4cf7-9b82-859a81f01726 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:11.762293 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d3758-c3db-4f6d-b094-7eb7ebecd1a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:11.762947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d3758-c3db-4f6d-b094-7eb7ebecd1a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.763223 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d3758-c3db-4f6d-b094-7eb7ebecd1a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.763497 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d3758-c3db-4f6d-b094-7eb7ebecd1a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.773948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d3758-c3db-4f6d-b094-7eb7ebecd1a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:11.775313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d3758-c3db-4f6d-b094-7eb7ebecd1a5 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:11.784695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1358/2718] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:11:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:11:11.850439 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65679e36-00b1-434b-a04e-85ab839e85a1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:11.852004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-65679e36-00b1-434b-a04e-85ab839e85a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:11.865549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65679e36-00b1-434b-a04e-85ab839e85a1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:11.870951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:11.871793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.872183 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.872697 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] PATH_INFO: `/v3/users/5dcf34766268492e819fbb971d394194` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.874170 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dcf34766268492e819fbb971d394194)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:11.876266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:11.899557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:11.935345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dcf34766268492e819fbb971d394194'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:11.944668 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dcf34766268492e819fbb971d394194'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:11.945479 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dcf34766268492e819fbb971d394194'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:11.958676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dcf34766268492e819fbb971d394194'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:11.966361 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dcf34766268492e819fbb971d394194'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:11.975115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1361/2719] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:11:11 2022] DELETE /identity/v3/users/5dcf34766268492e819fbb971d394194 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:11:11.979670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78de5a5-2936-40da-bf45-d77e1d5a4d20 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:11.979963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78de5a5-2936-40da-bf45-d77e1d5a4d20 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:11.980263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78de5a5-2936-40da-bf45-d77e1d5a4d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:11.999359 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f78de5a5-2936-40da-bf45-d77e1d5a4d20 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:12.000986 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f78de5a5-2936-40da-bf45-d77e1d5a4d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:12.023877 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:12.070741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1359/2720] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:11:12.080366 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:12.081908 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:12.168635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8df6b-66cb-45ac-82b4-1d40e6cc3d2e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:12.172275 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b6c08-8b00-41db-8714-8eed23b87969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:12.172952 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b6c08-8b00-41db-8714-8eed23b87969 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:12.173232 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b6c08-8b00-41db-8714-8eed23b87969 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:12.173508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b6c08-8b00-41db-8714-8eed23b87969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:12.184743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b6c08-8b00-41db-8714-8eed23b87969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:12.186287 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b6c08-8b00-41db-8714-8eed23b87969 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:12.196883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1362/2721] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:11:12.555232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667547d-88cb-43f7-bda0-74cc74053302 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:12.555707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667547d-88cb-43f7-bda0-74cc74053302 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:12.555846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667547d-88cb-43f7-bda0-74cc74053302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:12.587805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4667547d-88cb-43f7-bda0-74cc74053302 None None] MFA Rules not processed for user `1f2ac51306654b26ad063a6f559f3bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:12.589159 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4667547d-88cb-43f7-bda0-74cc74053302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:12.633573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b6c08-8b00-41db-8714-8eed23b87969 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:12.635159 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8c6b6c08-8b00-41db-8714-8eed23b87969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:12.652230 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b6c08-8b00-41db-8714-8eed23b87969 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:12.657371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:12.658065 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:12.658350 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:12.658621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] PATH_INFO: `/v3/projects/f7cc308152b04d2db04bd8ec2da31dec` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:12.659825 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7cc308152b04d2db04bd8ec2da31dec)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:12.662529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:12.672470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1360/2722] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:11:12.685207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667547d-88cb-43f7-bda0-74cc74053302 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:12.686676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4667547d-88cb-43f7-bda0-74cc74053302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:12.722277 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7cc308152b04d2db04bd8ec2da31dec', 'request_id': 'req-7c299f01-4f4e-402a-99b6-94be7534e8a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:12.730935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7cc308152b04d2db04bd8ec2da31dec', 'request_id': 'req-7c299f01-4f4e-402a-99b6-94be7534e8a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:12.737884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7cc308152b04d2db04bd8ec2da31dec', 'request_id': 'req-7c299f01-4f4e-402a-99b6-94be7534e8a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:12.749853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] The token cache is being invalidate because project f7cc308152b04d2db04bd8ec2da31dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:11:12.750138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:12.750760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7c299f01-4f4e-402a-99b6-94be7534e8a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:11:12.752339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1363/2723] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:11:12 2022] DELETE /identity/v3/projects/f7cc308152b04d2db04bd8ec2da31dec => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:11:12.761739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667547d-88cb-43f7-bda0-74cc74053302 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:12.825485 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8c32c-bb9b-4b8b-8a71-d11a0ff2e69b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:12.826163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8c32c-bb9b-4b8b-8a71-d11a0ff2e69b service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:12.826450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8c32c-bb9b-4b8b-8a71-d11a0ff2e69b service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:12.826730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8c32c-bb9b-4b8b-8a71-d11a0ff2e69b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:12.828738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b1c8c32c-bb9b-4b8b-8a71-d11a0ff2e69b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:12.861056 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:12.899146 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8c32c-bb9b-4b8b-8a71-d11a0ff2e69b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:12.900657 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8c32c-bb9b-4b8b-8a71-d11a0ff2e69b service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:12.912328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1361/2724] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:11:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:11:14.275300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6a8e-8661-4d72-b876-977c3238241d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:14.275858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6a8e-8661-4d72-b876-977c3238241d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:14.275858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa6a8e-8661-4d72-b876-977c3238241d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:14.294338 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-29aa6a8e-8661-4d72-b876-977c3238241d None None] MFA Rules not processed for user `1f2ac51306654b26ad063a6f559f3bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:14.295546 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-29aa6a8e-8661-4d72-b876-977c3238241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:14.302758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1364/2725] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:22.906123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=16 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:24.288822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=15,keystone:SELECT=76,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:33.763555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a8460-6f52-4081-bcd2-2b1f9ff45b87 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:33.764044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a8460-6f52-4081-bcd2-2b1f9ff45b87 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:33.764150 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a8460-6f52-4081-bcd2-2b1f9ff45b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:33.782499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7e4a8460-6f52-4081-bcd2-2b1f9ff45b87 None None] MFA Rules not processed for user `293d4ca894e24e4fb6c4e7aa14256d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:33.783722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7e4a8460-6f52-4081-bcd2-2b1f9ff45b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:33.850110 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1362/2726] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:34.874529 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb150f-1f1d-4a28-8f6e-f28c3c8cf9cc None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:34.874968 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb150f-1f1d-4a28-8f6e-f28c3c8cf9cc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:34.875207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb150f-1f1d-4a28-8f6e-f28c3c8cf9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:34.894717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-08cb150f-1f1d-4a28-8f6e-f28c3c8cf9cc None None] MFA Rules not processed for user `1f2ac51306654b26ad063a6f559f3bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:34.896866 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-08cb150f-1f1d-4a28-8f6e-f28c3c8cf9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:34.904427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1365/2727] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:11:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:35.143277 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a8460-6f52-4081-bcd2-2b1f9ff45b87 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:35.144832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7e4a8460-6f52-4081-bcd2-2b1f9ff45b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:35.217274 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a8460-6f52-4081-bcd2-2b1f9ff45b87 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:35.221832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066fce5-c35e-410e-8809-0723d5eb8f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:35.222487 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066fce5-c35e-410e-8809-0723d5eb8f6e service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:35.222772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066fce5-c35e-410e-8809-0723d5eb8f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:35.223051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066fce5-c35e-410e-8809-0723d5eb8f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:35.225958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e066fce5-c35e-410e-8809-0723d5eb8f6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:35.238237 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066fce5-c35e-410e-8809-0723d5eb8f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:35.241133 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066fce5-c35e-410e-8809-0723d5eb8f6e service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:35.250224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1363/2728] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:11:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:11:44.887864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:45.246589 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=51 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:11:56.382726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e6923-6514-4613-bff0-6af63b1fa746 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:56.383224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e6923-6514-4613-bff0-6af63b1fa746 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:56.383224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e6923-6514-4613-bff0-6af63b1fa746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:56.402318 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b02e6923-6514-4613-bff0-6af63b1fa746 None None] MFA Rules not processed for user `75f11eaa4b504d0aa4018fb4ffbbd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:11:56.403521 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b02e6923-6514-4613-bff0-6af63b1fa746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:56.463249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1366/2729] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 18:11:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:11:56.509449 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066fce5-c35e-410e-8809-0723d5eb8f6e service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:11:56.510854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e066fce5-c35e-410e-8809-0723d5eb8f6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:11:56.578606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066fce5-c35e-410e-8809-0723d5eb8f6e service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:11:56.583591 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e9330-11d7-4677-9beb-a16cbe52a3ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a26412cf9f64b1bbbd8e595b55bcebc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:11:56.584232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e9330-11d7-4677-9beb-a16cbe52a3ce service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:11:56.584605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e9330-11d7-4677-9beb-a16cbe52a3ce service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:11:56.584885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e9330-11d7-4677-9beb-a16cbe52a3ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:11:56.592837 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e9330-11d7-4677-9beb-a16cbe52a3ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:11:56.594392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e9330-11d7-4677-9beb-a16cbe52a3ce service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:11:56.602257 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1364/2730] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jul 21 18:11:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 18:12:06.455944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=25 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:12:06.599118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=32 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:12:11.870122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d4de0-7f6c-4f6c-bfe0-cdf7540c4669 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:12:11.870579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d4de0-7f6c-4f6c-bfe0-cdf7540c4669 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:12:11.870686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d4de0-7f6c-4f6c-bfe0-cdf7540c4669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:12:11.892591 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-619d4de0-7f6c-4f6c-bfe0-cdf7540c4669 None None] MFA Rules not processed for user `75f11eaa4b504d0aa4018fb4ffbbd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:12:11.893839 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-619d4de0-7f6c-4f6c-bfe0-cdf7540c4669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:12:11.901523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1367/2731] 10.222.0.57 () {58 vars in 967 bytes} [Thu Jul 21 18:12:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:12:11.914088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e9330-11d7-4677-9beb-a16cbe52a3ce service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:12:11.924916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e9330-11d7-4677-9beb-a16cbe52a3ce service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:12:11.930615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564b3bf-5b73-4979-ad79-d85839801175 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a26412cf9f64b1bbbd8e595b55bcebc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:12:11.931258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564b3bf-5b73-4979-ad79-d85839801175 service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:12:11.931540 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564b3bf-5b73-4979-ad79-d85839801175 service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:12:11.931816 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564b3bf-5b73-4979-ad79-d85839801175 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:12:11.940148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564b3bf-5b73-4979-ad79-d85839801175 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:12:11.942756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564b3bf-5b73-4979-ad79-d85839801175 service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:12:11.950777 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1365/2732] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jul 21 18:12:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 18:12:13.886911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d4de0-7f6c-4f6c-bfe0-cdf7540c4669 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:12:13.888441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-619d4de0-7f6c-4f6c-bfe0-cdf7540c4669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:12:13.965538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d4de0-7f6c-4f6c-bfe0-cdf7540c4669 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:12:13.970281 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:12:13.971158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:12:13.971441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:12:13.971722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] PATH_INFO: `/v3/users/1f2ac51306654b26ad063a6f559f3bfb` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:12:13.973188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2ac51306654b26ad063a6f559f3bfb)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:12:13.975149 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:12:14.024448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2ac51306654b26ad063a6f559f3bfb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:12:14.031753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2ac51306654b26ad063a6f559f3bfb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:12:14.032468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2ac51306654b26ad063a6f559f3bfb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:12:14.045184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2ac51306654b26ad063a6f559f3bfb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:12:14.052738 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8dd26baa-cf57-45c5-81ed-8f682140ee92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2ac51306654b26ad063a6f559f3bfb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:12:14.058616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1368/2733] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:12:13 2022] DELETE /identity/v3/users/1f2ac51306654b26ad063a6f559f3bfb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:12:14.236434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564b3bf-5b73-4979-ad79-d85839801175 service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:12:14.248456 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564b3bf-5b73-4979-ad79-d85839801175 service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:12:14.313371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:12:14.314232 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:12:14.314536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:12:14.314809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] PATH_INFO: `/v3/projects/7a1469ed86624fbe90592a6ded1740ad` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:12:14.316912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1469ed86624fbe90592a6ded1740ad)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:12:14.318812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:12:14.368181 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1469ed86624fbe90592a6ded1740ad', 'request_id': 'req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:12:14.374842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1469ed86624fbe90592a6ded1740ad', 'request_id': 'req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:12:14.380844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1469ed86624fbe90592a6ded1740ad', 'request_id': 'req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:12:14.392316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] The token cache is being invalidate because project 7a1469ed86624fbe90592a6ded1740ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:12:14.392614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:12:14.393252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:12:14.394957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1366/2734] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:12:14 2022] DELETE /identity/v3/projects/7a1469ed86624fbe90592a6ded1740ad => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:12:24.055079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:12:24.389159 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:28.098397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0d96-8358-495c-9dca-fddb0a635731 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:28.098906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0d96-8358-495c-9dca-fddb0a635731 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:28.099005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0d96-8358-495c-9dca-fddb0a635731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:28.116114 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f7ab0d96-8358-495c-9dca-fddb0a635731 None None] MFA Rules not processed for user `293d4ca894e24e4fb6c4e7aa14256d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:28.117483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f7ab0d96-8358-495c-9dca-fddb0a635731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:28.170951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1369/2735] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:28.482064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:28.483516 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:28.540790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af125e0-4da5-44b9-a2f8-e7f7c81ef4cd admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:28.544484 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30864672-8ed1-43bc-9bef-a80df3141ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:28.545142 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864672-8ed1-43bc-9bef-a80df3141ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:28.545424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864672-8ed1-43bc-9bef-a80df3141ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:28.545950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864672-8ed1-43bc-9bef-a80df3141ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:28.547745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-30864672-8ed1-43bc-9bef-a80df3141ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:28.558563 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30864672-8ed1-43bc-9bef-a80df3141ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:28.559976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30864672-8ed1-43bc-9bef-a80df3141ba2 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:28.567959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1367/2736] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:29.022517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab0d96-8358-495c-9dca-fddb0a635731 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:29.025146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f7ab0d96-8358-495c-9dca-fddb0a635731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:29.100451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab0d96-8358-495c-9dca-fddb0a635731 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:29.105482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9afc39-6d19-45e2-85a1-23722b0211a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:29.106124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9afc39-6d19-45e2-85a1-23722b0211a6 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:29.106409 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9afc39-6d19-45e2-85a1-23722b0211a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:29.106689 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9afc39-6d19-45e2-85a1-23722b0211a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:29.108477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8f9afc39-6d19-45e2-85a1-23722b0211a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:29.119845 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9afc39-6d19-45e2-85a1-23722b0211a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:29.121444 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9afc39-6d19-45e2-85a1-23722b0211a6 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:29.131096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1370/2737] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:31.726741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30864672-8ed1-43bc-9bef-a80df3141ba2 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:31.728807 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-30864672-8ed1-43bc-9bef-a80df3141ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:31.851026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30864672-8ed1-43bc-9bef-a80df3141ba2 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:31.858310 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b82ff-a0b7-4ce0-b730-a5bec5d9aa1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:31.858991 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b82ff-a0b7-4ce0-b730-a5bec5d9aa1f service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:31.859276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b82ff-a0b7-4ce0-b730-a5bec5d9aa1f service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:31.859556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b82ff-a0b7-4ce0-b730-a5bec5d9aa1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:31.861489 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3d0b82ff-a0b7-4ce0-b730-a5bec5d9aa1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:31.878207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b82ff-a0b7-4ce0-b730-a5bec5d9aa1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:31.879322 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b82ff-a0b7-4ce0-b730-a5bec5d9aa1f service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:31.899429 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1368/2738] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:38.205788 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9afc39-6d19-45e2-85a1-23722b0211a6 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:38.244030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9afc39-6d19-45e2-85a1-23722b0211a6 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:38.249701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87e821-4177-4995-9970-1ead5dd5fcee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:38.250354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87e821-4177-4995-9970-1ead5dd5fcee service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:38.250642 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87e821-4177-4995-9970-1ead5dd5fcee service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:38.250923 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87e821-4177-4995-9970-1ead5dd5fcee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:38.252746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ca87e821-4177-4995-9970-1ead5dd5fcee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:38.269791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87e821-4177-4995-9970-1ead5dd5fcee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:38.271645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87e821-4177-4995-9970-1ead5dd5fcee service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:38.281184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1371/2739] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:39.025556 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b82ff-a0b7-4ce0-b730-a5bec5d9aa1f service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:39.046690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b82ff-a0b7-4ce0-b730-a5bec5d9aa1f service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:39.052174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a1ea1-8abf-49d6-b4d9-ed2a15b36a14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:39.052850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a1ea1-8abf-49d6-b4d9-ed2a15b36a14 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:39.053131 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a1ea1-8abf-49d6-b4d9-ed2a15b36a14 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:39.053404 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a1ea1-8abf-49d6-b4d9-ed2a15b36a14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:39.055797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a85a1ea1-8abf-49d6-b4d9-ed2a15b36a14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:39.079781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a1ea1-8abf-49d6-b4d9-ed2a15b36a14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:39.081409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a1ea1-8abf-49d6-b4d9-ed2a15b36a14 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:39.091099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1369/2740] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:42.003030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87e821-4177-4995-9970-1ead5dd5fcee service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:42.014643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87e821-4177-4995-9970-1ead5dd5fcee service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:42.020519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65454a48-cfcd-45bf-8b06-48ab02f5547a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:42.021203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454a48-cfcd-45bf-8b06-48ab02f5547a service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:42.021490 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454a48-cfcd-45bf-8b06-48ab02f5547a service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:42.021771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454a48-cfcd-45bf-8b06-48ab02f5547a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:42.023542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-65454a48-cfcd-45bf-8b06-48ab02f5547a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:42.037156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65454a48-cfcd-45bf-8b06-48ab02f5547a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:42.038790 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65454a48-cfcd-45bf-8b06-48ab02f5547a service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:42.052086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1372/2741] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:47.833369 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a1ea1-8abf-49d6-b4d9-ed2a15b36a14 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:47.834775 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a85a1ea1-8abf-49d6-b4d9-ed2a15b36a14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:47.846744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a1ea1-8abf-49d6-b4d9-ed2a15b36a14 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:47.850296 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29db44-43a6-40ad-a0d2-ac3eb598502f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:47.850932 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29db44-43a6-40ad-a0d2-ac3eb598502f service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:47.851240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29db44-43a6-40ad-a0d2-ac3eb598502f service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:47.851517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29db44-43a6-40ad-a0d2-ac3eb598502f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:47.853252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bd29db44-43a6-40ad-a0d2-ac3eb598502f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:47.888222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:47.924499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29db44-43a6-40ad-a0d2-ac3eb598502f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:47.925943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29db44-43a6-40ad-a0d2-ac3eb598502f service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:47.935962 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1370/2742] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:50.971030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65454a48-cfcd-45bf-8b06-48ab02f5547a service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:50.973354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-65454a48-cfcd-45bf-8b06-48ab02f5547a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:50.991728 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65454a48-cfcd-45bf-8b06-48ab02f5547a service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:50.997460 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da1887-677f-46a9-b37a-40b6f2974fa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:50.998221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da1887-677f-46a9-b37a-40b6f2974fa7 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:50.998581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da1887-677f-46a9-b37a-40b6f2974fa7 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:50.998933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da1887-677f-46a9-b37a-40b6f2974fa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.001588 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-15da1887-677f-46a9-b37a-40b6f2974fa7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:51.017979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da1887-677f-46a9-b37a-40b6f2974fa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:51.019756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da1887-677f-46a9-b37a-40b6f2974fa7 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:51.036461 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1373/2743] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:51.045084 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29db44-43a6-40ad-a0d2-ac3eb598502f service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:51.046755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bd29db44-43a6-40ad-a0d2-ac3eb598502f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:51.059286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29db44-43a6-40ad-a0d2-ac3eb598502f service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:51.063842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:51.064879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.065161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.065434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] PATH_INFO: `/v3/users/2ea3ea36439143b8a55f181170efc14a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.067289 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea3ea36439143b8a55f181170efc14a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:51.068838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:51.120845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea3ea36439143b8a55f181170efc14a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:51.121507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea3ea36439143b8a55f181170efc14a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:51.122167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea3ea36439143b8a55f181170efc14a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:51.129128 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea3ea36439143b8a55f181170efc14a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:51.141478 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbfd365d-4f36-426e-8c28-2f78dad85f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea3ea36439143b8a55f181170efc14a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:51.154305 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1371/2744] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:13:51 2022] DELETE /identity/v3/users/2ea3ea36439143b8a55f181170efc14a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:51.336628 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da1887-677f-46a9-b37a-40b6f2974fa7 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:51.352708 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da1887-677f-46a9-b37a-40b6f2974fa7 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:51.387737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:51.412152 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:51.413054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.413346 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.413627 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] PATH_INFO: `/v3/projects/5161720524a4488aa42e383597d30144` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.414809 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5161720524a4488aa42e383597d30144)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:51.417327 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:51.465621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5161720524a4488aa42e383597d30144', 'request_id': 'req-2806b869-4230-43b0-b920-5919c9076bba'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:51.471584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5161720524a4488aa42e383597d30144', 'request_id': 'req-2806b869-4230-43b0-b920-5919c9076bba'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:51.479605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5161720524a4488aa42e383597d30144', 'request_id': 'req-2806b869-4230-43b0-b920-5919c9076bba'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:51.489643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] The token cache is being invalidate because project 5161720524a4488aa42e383597d30144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:13:51.489867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:51.490483 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2806b869-4230-43b0-b920-5919c9076bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:51.492004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1374/2745] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:13:51 2022] DELETE /identity/v3/projects/5161720524a4488aa42e383597d30144 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:13:51.498364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ea811-fcad-4fc3-8001-65695d1787db None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.498652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ea811-fcad-4fc3-8001-65695d1787db None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.498928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ea811-fcad-4fc3-8001-65695d1787db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.516286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c06ea811-fcad-4fc3-8001-65695d1787db None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:51.517867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c06ea811-fcad-4fc3-8001-65695d1787db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:51.576723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1372/2746] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:51.584366 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884efd68-be3e-4c1a-84ff-3f9ddf1463ad None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.584658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884efd68-be3e-4c1a-84ff-3f9ddf1463ad None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.584936 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884efd68-be3e-4c1a-84ff-3f9ddf1463ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.603933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-884efd68-be3e-4c1a-84ff-3f9ddf1463ad None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:51.605145 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-884efd68-be3e-4c1a-84ff-3f9ddf1463ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:51.612895 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1375/2747] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:51.616917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ea811-fcad-4fc3-8001-65695d1787db None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:51.627098 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ea811-fcad-4fc3-8001-65695d1787db None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:51.631569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516472a-5eb6-40a3-8920-94d6d3d7e269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:51.632214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516472a-5eb6-40a3-8920-94d6d3d7e269 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.632533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516472a-5eb6-40a3-8920-94d6d3d7e269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.632811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516472a-5eb6-40a3-8920-94d6d3d7e269 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.633535 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516472a-5eb6-40a3-8920-94d6d3d7e269 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:13:51.633841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516472a-5eb6-40a3-8920-94d6d3d7e269 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:51.635614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516472a-5eb6-40a3-8920-94d6d3d7e269 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:51.646805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1373/2748] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:13:51.651298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884efd68-be3e-4c1a-84ff-3f9ddf1463ad None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:51.660314 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884efd68-be3e-4c1a-84ff-3f9ddf1463ad None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:51.664850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72e663-cf09-4eb8-bde7-32df6cf007a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:51.665488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72e663-cf09-4eb8-bde7-32df6cf007a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.666308 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72e663-cf09-4eb8-bde7-32df6cf007a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.666582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72e663-cf09-4eb8-bde7-32df6cf007a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.667268 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72e663-cf09-4eb8-bde7-32df6cf007a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:51.669186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72e663-cf09-4eb8-bde7-32df6cf007a8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:51.694445 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1376/2749] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:13:51 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:13:51.698716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516472a-5eb6-40a3-8920-94d6d3d7e269 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:51.708499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516472a-5eb6-40a3-8920-94d6d3d7e269 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:51.735782 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:51.768378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e28a7-0160-48a0-b6c3-8f58fec01944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:51.768974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e28a7-0160-48a0-b6c3-8f58fec01944 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.769254 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e28a7-0160-48a0-b6c3-8f58fec01944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.769527 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e28a7-0160-48a0-b6c3-8f58fec01944 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.770221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e28a7-0160-48a0-b6c3-8f58fec01944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:51.772121 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e28a7-0160-48a0-b6c3-8f58fec01944 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:51.793811 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1374/2750] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:13:51 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:13:51.798153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72e663-cf09-4eb8-bde7-32df6cf007a8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:51.808903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72e663-cf09-4eb8-bde7-32df6cf007a8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:51.813505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7392f5-db0e-4baf-ba69-57f59045d31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:51.814154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7392f5-db0e-4baf-ba69-57f59045d31c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.814437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7392f5-db0e-4baf-ba69-57f59045d31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.814716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7392f5-db0e-4baf-ba69-57f59045d31c admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.815441 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7392f5-db0e-4baf-ba69-57f59045d31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:51.817113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7392f5-db0e-4baf-ba69-57f59045d31c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:51.827406 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1377/2751] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:13:51.831141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e28a7-0160-48a0-b6c3-8f58fec01944 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:51.841562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e28a7-0160-48a0-b6c3-8f58fec01944 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:51.846283 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84a2d8-5d67-40b2-ad26-0445eb247a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:51.846966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84a2d8-5d67-40b2-ad26-0445eb247a6e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.847249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84a2d8-5d67-40b2-ad26-0445eb247a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.847523 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84a2d8-5d67-40b2-ad26-0445eb247a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.848249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84a2d8-5d67-40b2-ad26-0445eb247a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:51.849869 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84a2d8-5d67-40b2-ad26-0445eb247a6e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:51.860457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1375/2752] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:13:51.864701 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7392f5-db0e-4baf-ba69-57f59045d31c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:51.874580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7392f5-db0e-4baf-ba69-57f59045d31c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:51.879117 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee51c-dcd3-4959-92cf-620683547628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:51.880009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee51c-dcd3-4959-92cf-620683547628 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.880289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee51c-dcd3-4959-92cf-620683547628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.880595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee51c-dcd3-4959-92cf-620683547628 admin admin] PATH_INFO: `/v3/projects/2a250b03982842529721ef841b4d122e/users/61a5bf73b5ba47cc82fc41e30620fbfd/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.896582 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aee51c-dcd3-4959-92cf-620683547628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a250b03982842529721ef841b4d122e, user_id=61a5bf73b5ba47cc82fc41e30620fbfd, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:51.899206 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aee51c-dcd3-4959-92cf-620683547628 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:51.912250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1378/2753] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:13:51 2022] PUT /identity/v3/projects/2a250b03982842529721ef841b4d122e/users/61a5bf73b5ba47cc82fc41e30620fbfd/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:51.924224 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c0d6b-fc8f-4d17-b7a7-d3b64bbede91 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:51.924561 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c0d6b-fc8f-4d17-b7a7-d3b64bbede91 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:51.924842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c0d6b-fc8f-4d17-b7a7-d3b64bbede91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:51.956200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-bc6c0d6b-fc8f-4d17-b7a7-d3b64bbede91 None None] MFA Rules not processed for user `61a5bf73b5ba47cc82fc41e30620fbfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:51.957457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bc6c0d6b-fc8f-4d17-b7a7-d3b64bbede91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:52.019958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1376/2754] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:52.031392 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee51c-dcd3-4959-92cf-620683547628 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:52.033168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-53aee51c-dcd3-4959-92cf-620683547628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:52.093265 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:52.110251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee51c-dcd3-4959-92cf-620683547628 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:52.115674 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac3fe0-1e98-4ffd-ab72-b56f4c3f1580 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:52.116339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3fe0-1e98-4ffd-ab72-b56f4c3f1580 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:52.116623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3fe0-1e98-4ffd-ab72-b56f4c3f1580 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:52.116893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3fe0-1e98-4ffd-ab72-b56f4c3f1580 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:52.124610 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac3fe0-1e98-4ffd-ab72-b56f4c3f1580 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:52.126179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac3fe0-1e98-4ffd-ab72-b56f4c3f1580 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:52.135424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1379/2755] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:52.342939 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c0d6b-fc8f-4d17-b7a7-d3b64bbede91 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:52.404974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c0d6b-fc8f-4d17-b7a7-d3b64bbede91 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:52.409424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:52.410292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:52.410569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:52.410840 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] PATH_INFO: `/v3/users/61a5bf73b5ba47cc82fc41e30620fbfd` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:52.412001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a5bf73b5ba47cc82fc41e30620fbfd)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:52.413959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:52.454229 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:52.462552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a5bf73b5ba47cc82fc41e30620fbfd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:52.463270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a5bf73b5ba47cc82fc41e30620fbfd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:52.464167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a5bf73b5ba47cc82fc41e30620fbfd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:52.471170 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a5bf73b5ba47cc82fc41e30620fbfd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:52.482815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a5bf73b5ba47cc82fc41e30620fbfd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:52.495709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1377/2756] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:13:52 2022] DELETE /identity/v3/users/61a5bf73b5ba47cc82fc41e30620fbfd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:52.500251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec97885-ee91-439c-87da-6b5bffbef1ac None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:52.500577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec97885-ee91-439c-87da-6b5bffbef1ac None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:52.500857 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec97885-ee91-439c-87da-6b5bffbef1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:52.518586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4ec97885-ee91-439c-87da-6b5bffbef1ac None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:52.519776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4ec97885-ee91-439c-87da-6b5bffbef1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:52.580387 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1380/2757] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:52.590042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:52.591432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:52.648693 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50e6d6-e84b-4899-9271-9c362fe3fde6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:52.652467 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243868a7-adb3-4734-a426-951f24fb0c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:52.653104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243868a7-adb3-4734-a426-951f24fb0c3f service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:52.653384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243868a7-adb3-4734-a426-951f24fb0c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:52.653656 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243868a7-adb3-4734-a426-951f24fb0c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:52.662542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243868a7-adb3-4734-a426-951f24fb0c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:52.663957 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243868a7-adb3-4734-a426-951f24fb0c3f service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:52.673681 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1378/2758] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:52.943386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec97885-ee91-439c-87da-6b5bffbef1ac None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:52.944872 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4ec97885-ee91-439c-87da-6b5bffbef1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:52.956578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec97885-ee91-439c-87da-6b5bffbef1ac None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:52.961632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:52.962555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:52.962797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:52.963078 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] PATH_INFO: `/v3/projects/2a250b03982842529721ef841b4d122e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:52.964258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a250b03982842529721ef841b4d122e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:52.966190 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.015034 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a250b03982842529721ef841b4d122e', 'request_id': 'req-667b1a2e-171c-4a9e-8e36-e24b938892f9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:53.020978 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a250b03982842529721ef841b4d122e', 'request_id': 'req-667b1a2e-171c-4a9e-8e36-e24b938892f9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:53.028177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a250b03982842529721ef841b4d122e', 'request_id': 'req-667b1a2e-171c-4a9e-8e36-e24b938892f9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:53.038285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] The token cache is being invalidate because project 2a250b03982842529721ef841b4d122e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:13:53.038858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:53.039326 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-667b1a2e-171c-4a9e-8e36-e24b938892f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:53.041752 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1381/2759] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:13:52 2022] DELETE /identity/v3/projects/2a250b03982842529721ef841b4d122e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:53.047722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c36fe6-bd24-4eeb-b7ac-a176bef7de4a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.048003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c36fe6-bd24-4eeb-b7ac-a176bef7de4a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.048286 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c36fe6-bd24-4eeb-b7ac-a176bef7de4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.066525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-55c36fe6-bd24-4eeb-b7ac-a176bef7de4a None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:53.067736 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-55c36fe6-bd24-4eeb-b7ac-a176bef7de4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:53.126885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1379/2760] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:53.135430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cb4f8-0091-4b0f-b97d-4a48e3d0ccce None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.135724 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cb4f8-0091-4b0f-b97d-4a48e3d0ccce None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.136009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cb4f8-0091-4b0f-b97d-4a48e3d0ccce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.154944 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d47cb4f8-0091-4b0f-b97d-4a48e3d0ccce None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:53.156161 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d47cb4f8-0091-4b0f-b97d-4a48e3d0ccce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:53.164025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1382/2761] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:53.168493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c36fe6-bd24-4eeb-b7ac-a176bef7de4a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.178480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c36fe6-bd24-4eeb-b7ac-a176bef7de4a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.183685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8d5e0-2812-426a-9fb3-599105e08b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.184364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8d5e0-2812-426a-9fb3-599105e08b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.184649 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8d5e0-2812-426a-9fb3-599105e08b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.184920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8d5e0-2812-426a-9fb3-599105e08b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.185646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8d5e0-2812-426a-9fb3-599105e08b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:13:53.185952 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8d5e0-2812-426a-9fb3-599105e08b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.187521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8d5e0-2812-426a-9fb3-599105e08b3a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.199548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1380/2762] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:13:53.203793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47cb4f8-0091-4b0f-b97d-4a48e3d0ccce None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.213123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47cb4f8-0091-4b0f-b97d-4a48e3d0ccce None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.217770 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bece425-96ee-4ab4-8a50-65d2003ee9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.218415 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bece425-96ee-4ab4-8a50-65d2003ee9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.218703 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bece425-96ee-4ab4-8a50-65d2003ee9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.218983 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bece425-96ee-4ab4-8a50-65d2003ee9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.219665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bece425-96ee-4ab4-8a50-65d2003ee9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.221581 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bece425-96ee-4ab4-8a50-65d2003ee9d3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.246616 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1383/2763] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:13:53 2022] POST /identity/v3/projects => generated 365 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:13:53.250628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8d5e0-2812-426a-9fb3-599105e08b3a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.261040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8d5e0-2812-426a-9fb3-599105e08b3a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.303435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:53.318894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b54981-541c-4a35-be7a-d030f063f6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.319555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b54981-541c-4a35-be7a-d030f063f6cf admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.319834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b54981-541c-4a35-be7a-d030f063f6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.320111 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b54981-541c-4a35-be7a-d030f063f6cf admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.320828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b54981-541c-4a35-be7a-d030f063f6cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.323509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b54981-541c-4a35-be7a-d030f063f6cf admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.345331 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1381/2764] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:13:53 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:13:53.347851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bece425-96ee-4ab4-8a50-65d2003ee9d3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.358542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bece425-96ee-4ab4-8a50-65d2003ee9d3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.363305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a94d5-71c2-4566-9192-0c24783241fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.363957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a94d5-71c2-4566-9192-0c24783241fd admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.364241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a94d5-71c2-4566-9192-0c24783241fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.364903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a94d5-71c2-4566-9192-0c24783241fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.365634 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a94d5-71c2-4566-9192-0c24783241fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.367234 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a94d5-71c2-4566-9192-0c24783241fd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.378666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1384/2765] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:13:53.382514 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b54981-541c-4a35-be7a-d030f063f6cf admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.393263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b54981-541c-4a35-be7a-d030f063f6cf admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.397781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9d5d8-546b-4ec2-a028-28aaf61d3f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.398432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9d5d8-546b-4ec2-a028-28aaf61d3f37 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.398714 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9d5d8-546b-4ec2-a028-28aaf61d3f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.398990 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9d5d8-546b-4ec2-a028-28aaf61d3f37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.399709 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9d5d8-546b-4ec2-a028-28aaf61d3f37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.401371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9d5d8-546b-4ec2-a028-28aaf61d3f37 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.411081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1382/2766] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:13:53.414978 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a94d5-71c2-4566-9192-0c24783241fd admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.425851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a94d5-71c2-4566-9192-0c24783241fd admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.430515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c589c-c1f4-4b69-a9a8-7727bc5f6709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.432044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c589c-c1f4-4b69-a9a8-7727bc5f6709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.432449 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c589c-c1f4-4b69-a9a8-7727bc5f6709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.432737 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c589c-c1f4-4b69-a9a8-7727bc5f6709 admin admin] PATH_INFO: `/v3/projects/bab018c6819a4e4894f5d28b92d6c5d6/users/06620a6be16740c5850dd82cd76b17c0/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.447637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c589c-c1f4-4b69-a9a8-7727bc5f6709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab018c6819a4e4894f5d28b92d6c5d6, user_id=06620a6be16740c5850dd82cd76b17c0, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.450325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c589c-c1f4-4b69-a9a8-7727bc5f6709 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.462795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1385/2767] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:13:53 2022] PUT /identity/v3/projects/bab018c6819a4e4894f5d28b92d6c5d6/users/06620a6be16740c5850dd82cd76b17c0/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:53.473072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5aa9-bdcb-4a0e-b4a1-b92794ca634e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.473349 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5aa9-bdcb-4a0e-b4a1-b92794ca634e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.473628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5aa9-bdcb-4a0e-b4a1-b92794ca634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.499823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8cfd5aa9-bdcb-4a0e-b4a1-b92794ca634e None None] MFA Rules not processed for user `06620a6be16740c5850dd82cd76b17c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:53.501081 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8cfd5aa9-bdcb-4a0e-b4a1-b92794ca634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:53.566243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1383/2768] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:53.570678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c589c-c1f4-4b69-a9a8-7727bc5f6709 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.579443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:53.635303 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c589c-c1f4-4b69-a9a8-7727bc5f6709 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.639841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3da9b-0852-4e4b-b957-9fd0a33f8033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.640528 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3da9b-0852-4e4b-b957-9fd0a33f8033 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.640813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3da9b-0852-4e4b-b957-9fd0a33f8033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.641090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3da9b-0852-4e4b-b957-9fd0a33f8033 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.641803 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3da9b-0852-4e4b-b957-9fd0a33f8033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.643702 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3da9b-0852-4e4b-b957-9fd0a33f8033 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.669579 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1386/2769] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:13:53 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:13:53.673879 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd5aa9-bdcb-4a0e-b4a1-b92794ca634e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.685521 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd5aa9-bdcb-4a0e-b4a1-b92794ca634e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.741687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c4867-798d-4c41-8114-b9376b42113d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.742562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c4867-798d-4c41-8114-b9376b42113d admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.742846 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c4867-798d-4c41-8114-b9376b42113d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.743124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c4867-798d-4c41-8114-b9376b42113d admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.744470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c4867-798d-4c41-8114-b9376b42113d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.746381 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c4867-798d-4c41-8114-b9376b42113d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.766737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1384/2770] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:13:53 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:13:53.770557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3da9b-0852-4e4b-b957-9fd0a33f8033 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.781272 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3da9b-0852-4e4b-b957-9fd0a33f8033 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.785888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5734d-e9dc-43ea-b21e-ede6f7e5b676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.786536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5734d-e9dc-43ea-b21e-ede6f7e5b676 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.786822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5734d-e9dc-43ea-b21e-ede6f7e5b676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.787100 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5734d-e9dc-43ea-b21e-ede6f7e5b676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.787823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5734d-e9dc-43ea-b21e-ede6f7e5b676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.789452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5734d-e9dc-43ea-b21e-ede6f7e5b676 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.800774 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1387/2771] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:13:53.804756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c4867-798d-4c41-8114-b9376b42113d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.815726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c4867-798d-4c41-8114-b9376b42113d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.820335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7678d-446d-488d-8875-6e096a8d0418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.821211 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7678d-446d-488d-8875-6e096a8d0418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.821493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7678d-446d-488d-8875-6e096a8d0418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.821859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7678d-446d-488d-8875-6e096a8d0418 admin admin] PATH_INFO: `/v3/projects/6eca9bf52f104c608899677d5471fa4e/users/1a92f25bbc774e61bae328b0f204111c/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.836330 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7678d-446d-488d-8875-6e096a8d0418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eca9bf52f104c608899677d5471fa4e, user_id=1a92f25bbc774e61bae328b0f204111c, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.839144 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7678d-446d-488d-8875-6e096a8d0418 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.851702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1385/2772] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:13:53 2022] PUT /identity/v3/projects/6eca9bf52f104c608899677d5471fa4e/users/1a92f25bbc774e61bae328b0f204111c/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:53.856342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5734d-e9dc-43ea-b21e-ede6f7e5b676 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.866347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5734d-e9dc-43ea-b21e-ede6f7e5b676 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.924371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed9a7e-2b5a-4536-9650-b2fba5a7a6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.924967 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed9a7e-2b5a-4536-9650-b2fba5a7a6fe admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.925256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed9a7e-2b5a-4536-9650-b2fba5a7a6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.925540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed9a7e-2b5a-4536-9650-b2fba5a7a6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.926271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed9a7e-2b5a-4536-9650-b2fba5a7a6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.927881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed9a7e-2b5a-4536-9650-b2fba5a7a6fe admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.938520 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1388/2773] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:13:53.942908 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7678d-446d-488d-8875-6e096a8d0418 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:53.953698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7678d-446d-488d-8875-6e096a8d0418 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:53.958343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0218fe2-90cf-453a-a2d8-3a1e7dbfacd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:53.959250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0218fe2-90cf-453a-a2d8-3a1e7dbfacd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:53.959534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0218fe2-90cf-453a-a2d8-3a1e7dbfacd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:53.959812 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0218fe2-90cf-453a-a2d8-3a1e7dbfacd9 admin admin] PATH_INFO: `/v3/projects/6eca9bf52f104c608899677d5471fa4e/users/1a92f25bbc774e61bae328b0f204111c/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:53.961853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0218fe2-90cf-453a-a2d8-3a1e7dbfacd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eca9bf52f104c608899677d5471fa4e, user_id=1a92f25bbc774e61bae328b0f204111c, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:53.965030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0218fe2-90cf-453a-a2d8-3a1e7dbfacd9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:53.975956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1386/2774] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:13:53 2022] PUT /identity/v3/projects/6eca9bf52f104c608899677d5471fa4e/users/1a92f25bbc774e61bae328b0f204111c/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:54.034641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ac638-b4ed-4637-a6ca-86222e89c401 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:54.034937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ac638-b4ed-4637-a6ca-86222e89c401 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:54.035224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ac638-b4ed-4637-a6ca-86222e89c401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:54.063317 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ed4ac638-b4ed-4637-a6ca-86222e89c401 None None] MFA Rules not processed for user `1a92f25bbc774e61bae328b0f204111c`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:54.064682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ed4ac638-b4ed-4637-a6ca-86222e89c401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:54.133858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1389/2775] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:54.146311 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0218fe2-90cf-453a-a2d8-3a1e7dbfacd9 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:54.148028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b0218fe2-90cf-453a-a2d8-3a1e7dbfacd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:54.158462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:54.208238 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0218fe2-90cf-453a-a2d8-3a1e7dbfacd9 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:54.211795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20837ec1-a0a4-47aa-97ed-25ff561d5fc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:54.212592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20837ec1-a0a4-47aa-97ed-25ff561d5fc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:54.212872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20837ec1-a0a4-47aa-97ed-25ff561d5fc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:54.213145 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20837ec1-a0a4-47aa-97ed-25ff561d5fc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:54.221859 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20837ec1-a0a4-47aa-97ed-25ff561d5fc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:54.223262 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20837ec1-a0a4-47aa-97ed-25ff561d5fc7 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:54.232791 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1387/2776] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:54.275635 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ac638-b4ed-4637-a6ca-86222e89c401 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:54.277091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ed4ac638-b4ed-4637-a6ca-86222e89c401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:54.283874 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:54.291294 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ac638-b4ed-4637-a6ca-86222e89c401 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:54.295553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d22da-8d77-4a86-8747-ed25761eccf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:54.296201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d22da-8d77-4a86-8747-ed25761eccf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:54.296514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d22da-8d77-4a86-8747-ed25761eccf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:54.296799 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d22da-8d77-4a86-8747-ed25761eccf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:54.356565 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d22da-8d77-4a86-8747-ed25761eccf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:54.358017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d22da-8d77-4a86-8747-ed25761eccf3 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:54.366924 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1390/2777] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:57.635573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20837ec1-a0a4-47aa-97ed-25ff561d5fc7 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:57.700378 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20837ec1-a0a4-47aa-97ed-25ff561d5fc7 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:57.705031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:57.705895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:57.706178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:57.706454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] PATH_INFO: `/v3/users/06620a6be16740c5850dd82cd76b17c0` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:57.707630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=06620a6be16740c5850dd82cd76b17c0)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:57.709582 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:57.755003 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06620a6be16740c5850dd82cd76b17c0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:57.755646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06620a6be16740c5850dd82cd76b17c0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:57.756247 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06620a6be16740c5850dd82cd76b17c0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:57.763125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06620a6be16740c5850dd82cd76b17c0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:57.775370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06620a6be16740c5850dd82cd76b17c0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:57.788308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1388/2778] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:13:57 2022] DELETE /identity/v3/users/06620a6be16740c5850dd82cd76b17c0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:57.793463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63775367-114c-4cba-bb02-a183890e50d7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:57.793928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63775367-114c-4cba-bb02-a183890e50d7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:57.794020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63775367-114c-4cba-bb02-a183890e50d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:57.820807 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-63775367-114c-4cba-bb02-a183890e50d7 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:57.822189 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-63775367-114c-4cba-bb02-a183890e50d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:57.879729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1391/2779] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:57.890215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:57.891641 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:57.921091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:57.973417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44bbe3-13d9-4dc4-9a8f-5653254608ac admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:57.978958 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d9aa0-bd5a-4dc7-94d2-35886c6d1b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:57.979608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d9aa0-bd5a-4dc7-94d2-35886c6d1b17 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:57.979891 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d9aa0-bd5a-4dc7-94d2-35886c6d1b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:57.980168 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d9aa0-bd5a-4dc7-94d2-35886c6d1b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:57.989197 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d9aa0-bd5a-4dc7-94d2-35886c6d1b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:57.990619 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d9aa0-bd5a-4dc7-94d2-35886c6d1b17 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:58.000403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1389/2780] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:13:58.273030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63775367-114c-4cba-bb02-a183890e50d7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:58.274509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-63775367-114c-4cba-bb02-a183890e50d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:58.290113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63775367-114c-4cba-bb02-a183890e50d7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:58.296852 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:58.297716 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:58.298055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:58.298252 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] PATH_INFO: `/v3/projects/bab018c6819a4e4894f5d28b92d6c5d6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:58.299431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab018c6819a4e4894f5d28b92d6c5d6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:58.301354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:58.348197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab018c6819a4e4894f5d28b92d6c5d6', 'request_id': 'req-4f0909dc-a672-4a3d-8fa9-d59178e28b92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.354447 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab018c6819a4e4894f5d28b92d6c5d6', 'request_id': 'req-4f0909dc-a672-4a3d-8fa9-d59178e28b92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.361709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab018c6819a4e4894f5d28b92d6c5d6', 'request_id': 'req-4f0909dc-a672-4a3d-8fa9-d59178e28b92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.372241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] The token cache is being invalidate because project bab018c6819a4e4894f5d28b92d6c5d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:13:58.372555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.373177 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.374686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1392/2781] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:13:58 2022] DELETE /identity/v3/projects/bab018c6819a4e4894f5d28b92d6c5d6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:58.378610 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d9aa0-bd5a-4dc7-94d2-35886c6d1b17 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:58.388916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d9aa0-bd5a-4dc7-94d2-35886c6d1b17 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:58.444278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:58.445165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:58.445446 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:58.445720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] PATH_INFO: `/v3/users/1a92f25bbc774e61bae328b0f204111c` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:58.447555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a92f25bbc774e61bae328b0f204111c)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:58.449486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:58.494120 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a92f25bbc774e61bae328b0f204111c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.494769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a92f25bbc774e61bae328b0f204111c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.495367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a92f25bbc774e61bae328b0f204111c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.502139 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a92f25bbc774e61bae328b0f204111c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.514689 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-73d08de1-e804-45a2-8942-06b4c67c3970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a92f25bbc774e61bae328b0f204111c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.526912 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1390/2782] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:13:58 2022] DELETE /identity/v3/users/1a92f25bbc774e61bae328b0f204111c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:58.788013 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:58.789465 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:58.803354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0909dc-a672-4a3d-8fa9-d59178e28b92 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:58.816115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:58.862755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:58.863613 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:58.863897 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:58.864224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] PATH_INFO: `/v3/projects/6eca9bf52f104c608899677d5471fa4e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:58.865486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eca9bf52f104c608899677d5471fa4e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:58.867420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:58.915043 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eca9bf52f104c608899677d5471fa4e', 'request_id': 'req-4d673420-09ca-4521-98c0-4f35c3c50bff'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.922469 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eca9bf52f104c608899677d5471fa4e', 'request_id': 'req-4d673420-09ca-4521-98c0-4f35c3c50bff'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.932216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eca9bf52f104c608899677d5471fa4e', 'request_id': 'req-4d673420-09ca-4521-98c0-4f35c3c50bff'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.942991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] The token cache is being invalidate because project 6eca9bf52f104c608899677d5471fa4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:13:58.943285 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.943919 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4d673420-09ca-4521-98c0-4f35c3c50bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:13:58.945488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1393/2783] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:13:58 2022] DELETE /identity/v3/projects/6eca9bf52f104c608899677d5471fa4e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:58.950355 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99e84a-d84b-4a40-a7cc-c80aa3a85a58 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:58.950790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99e84a-d84b-4a40-a7cc-c80aa3a85a58 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:58.950951 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99e84a-d84b-4a40-a7cc-c80aa3a85a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:58.969282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0c99e84a-d84b-4a40-a7cc-c80aa3a85a58 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:58.970474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0c99e84a-d84b-4a40-a7cc-c80aa3a85a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:58.997458 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:59.033749 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1391/2784] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:59.042072 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6af1c-7795-4161-a480-a951abe791cb None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.042362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6af1c-7795-4161-a480-a951abe791cb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.042641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6af1c-7795-4161-a480-a951abe791cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.061269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-26a6af1c-7795-4161-a480-a951abe791cb None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:59.062464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-26a6af1c-7795-4161-a480-a951abe791cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:59.070298 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1394/2785] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:13:59.074409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99e84a-d84b-4a40-a7cc-c80aa3a85a58 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.084001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99e84a-d84b-4a40-a7cc-c80aa3a85a58 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.088626 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d371268-8c4e-4fd3-9311-13f2595c405e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.089272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d371268-8c4e-4fd3-9311-13f2595c405e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.089548 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d371268-8c4e-4fd3-9311-13f2595c405e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.089822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d371268-8c4e-4fd3-9311-13f2595c405e admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.090580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d371268-8c4e-4fd3-9311-13f2595c405e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:13:59.090887 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d371268-8c4e-4fd3-9311-13f2595c405e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.092486 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d371268-8c4e-4fd3-9311-13f2595c405e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.108418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1392/2786] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:13:59.110076 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6af1c-7795-4161-a480-a951abe791cb None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.120084 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6af1c-7795-4161-a480-a951abe791cb None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.124977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5205c-96de-4ebb-b9b6-a38f9ec87561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.125638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5205c-96de-4ebb-b9b6-a38f9ec87561 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.125921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5205c-96de-4ebb-b9b6-a38f9ec87561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.126195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5205c-96de-4ebb-b9b6-a38f9ec87561 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.126899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5205c-96de-4ebb-b9b6-a38f9ec87561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.128833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5205c-96de-4ebb-b9b6-a38f9ec87561 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.154651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1395/2787] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:13:59 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:13:59.158953 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d371268-8c4e-4fd3-9311-13f2595c405e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.169049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d371268-8c4e-4fd3-9311-13f2595c405e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.227215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2e24b-01f1-4e5b-bf4d-6ad2ee254778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.228073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2e24b-01f1-4e5b-bf4d-6ad2ee254778 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.228387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2e24b-01f1-4e5b-bf4d-6ad2ee254778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.228660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2e24b-01f1-4e5b-bf4d-6ad2ee254778 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.229356 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2e24b-01f1-4e5b-bf4d-6ad2ee254778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.231272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2e24b-01f1-4e5b-bf4d-6ad2ee254778 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.250999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1393/2788] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:13:59 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:13:59.255213 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5205c-96de-4ebb-b9b6-a38f9ec87561 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.266171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5205c-96de-4ebb-b9b6-a38f9ec87561 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.271060 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46f7e9-1394-4c29-ab08-4e52f2a43e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.271722 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f7e9-1394-4c29-ab08-4e52f2a43e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.272008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f7e9-1394-4c29-ab08-4e52f2a43e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.272289 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f7e9-1394-4c29-ab08-4e52f2a43e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.273074 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46f7e9-1394-4c29-ab08-4e52f2a43e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.274678 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46f7e9-1394-4c29-ab08-4e52f2a43e00 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.285274 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1396/2789] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:13:59.289021 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2e24b-01f1-4e5b-bf4d-6ad2ee254778 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.299085 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2e24b-01f1-4e5b-bf4d-6ad2ee254778 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.303704 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dba539-da00-4b3f-bbb4-2c6c4baf53ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.304385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba539-da00-4b3f-bbb4-2c6c4baf53ad admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.304671 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba539-da00-4b3f-bbb4-2c6c4baf53ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.304950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba539-da00-4b3f-bbb4-2c6c4baf53ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.305678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dba539-da00-4b3f-bbb4-2c6c4baf53ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.307799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dba539-da00-4b3f-bbb4-2c6c4baf53ad admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.317657 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1394/2790] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:13:59.322142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46f7e9-1394-4c29-ab08-4e52f2a43e00 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.332519 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46f7e9-1394-4c29-ab08-4e52f2a43e00 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.337106 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04230902-c92a-43ea-ac87-53ad93c82fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.337980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04230902-c92a-43ea-ac87-53ad93c82fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.338263 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04230902-c92a-43ea-ac87-53ad93c82fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.338543 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04230902-c92a-43ea-ac87-53ad93c82fa7 admin admin] PATH_INFO: `/v3/projects/0908c2c18fb846448d82bef9a0225e73/users/89d5c9de65e8454abf456bb80ff8521a/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.352881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04230902-c92a-43ea-ac87-53ad93c82fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0908c2c18fb846448d82bef9a0225e73, user_id=89d5c9de65e8454abf456bb80ff8521a, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.355730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04230902-c92a-43ea-ac87-53ad93c82fa7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.369138 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1397/2791] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:13:59 2022] PUT /identity/v3/projects/0908c2c18fb846448d82bef9a0225e73/users/89d5c9de65e8454abf456bb80ff8521a/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:59.379255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dbcaa-3993-4bcc-891a-d3713be4de02 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.379551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dbcaa-3993-4bcc-891a-d3713be4de02 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.379832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dbcaa-3993-4bcc-891a-d3713be4de02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.406956 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6d0dbcaa-3993-4bcc-891a-d3713be4de02 None None] MFA Rules not processed for user `89d5c9de65e8454abf456bb80ff8521a`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:59.408175 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6d0dbcaa-3993-4bcc-891a-d3713be4de02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:59.471396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1395/2792] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:13:59.476033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04230902-c92a-43ea-ac87-53ad93c82fa7 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.521893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:59.541928 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04230902-c92a-43ea-ac87-53ad93c82fa7 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.547353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d1b80-0080-4bd9-b2ba-86b55352e462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.548096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d1b80-0080-4bd9-b2ba-86b55352e462 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.548512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d1b80-0080-4bd9-b2ba-86b55352e462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.548878 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d1b80-0080-4bd9-b2ba-86b55352e462 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.549667 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d1b80-0080-4bd9-b2ba-86b55352e462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.551650 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d1b80-0080-4bd9-b2ba-86b55352e462 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.576363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1398/2793] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:13:59 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:13:59.580492 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dbcaa-3993-4bcc-891a-d3713be4de02 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.591794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dbcaa-3993-4bcc-891a-d3713be4de02 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.615091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:13:59.647602 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2e9c4-402a-4b6c-b738-24bb0d9a9288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.648305 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2e9c4-402a-4b6c-b738-24bb0d9a9288 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.648590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2e9c4-402a-4b6c-b738-24bb0d9a9288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.648872 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2e9c4-402a-4b6c-b738-24bb0d9a9288 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.649564 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2e9c4-402a-4b6c-b738-24bb0d9a9288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.652180 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2e9c4-402a-4b6c-b738-24bb0d9a9288 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.671403 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1396/2794] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:13:59 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:13:59.675470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d1b80-0080-4bd9-b2ba-86b55352e462 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.685662 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d1b80-0080-4bd9-b2ba-86b55352e462 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.690201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a3f10-6a0f-4ec5-880e-78ddec5148a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.690851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a3f10-6a0f-4ec5-880e-78ddec5148a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.691140 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a3f10-6a0f-4ec5-880e-78ddec5148a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.691420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a3f10-6a0f-4ec5-880e-78ddec5148a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.692156 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a3f10-6a0f-4ec5-880e-78ddec5148a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.693804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a3f10-6a0f-4ec5-880e-78ddec5148a2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.704884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1399/2795] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:13:59.708842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2e9c4-402a-4b6c-b738-24bb0d9a9288 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.719248 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2e9c4-402a-4b6c-b738-24bb0d9a9288 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.723796 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b4a29-c286-4dcd-8651-b3eed84c5177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.724697 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b4a29-c286-4dcd-8651-b3eed84c5177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.724977 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b4a29-c286-4dcd-8651-b3eed84c5177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.725251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b4a29-c286-4dcd-8651-b3eed84c5177 admin admin] PATH_INFO: `/v3/projects/801d840f4b2940dc90067c09c271aa78/users/e1d3c8a88186413c89c3ec44b9ecb17c/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.741027 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b4a29-c286-4dcd-8651-b3eed84c5177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801d840f4b2940dc90067c09c271aa78, user_id=e1d3c8a88186413c89c3ec44b9ecb17c, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.743813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b4a29-c286-4dcd-8651-b3eed84c5177 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.756612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1397/2796] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:13:59 2022] PUT /identity/v3/projects/801d840f4b2940dc90067c09c271aa78/users/e1d3c8a88186413c89c3ec44b9ecb17c/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:13:59.760972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a3f10-6a0f-4ec5-880e-78ddec5148a2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.771220 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a3f10-6a0f-4ec5-880e-78ddec5148a2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.826947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85041b6a-126a-46c8-8295-0c1e38c07184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.827540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85041b6a-126a-46c8-8295-0c1e38c07184 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.827811 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85041b6a-126a-46c8-8295-0c1e38c07184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.828121 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85041b6a-126a-46c8-8295-0c1e38c07184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.828942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85041b6a-126a-46c8-8295-0c1e38c07184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.830544 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85041b6a-126a-46c8-8295-0c1e38c07184 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.840896 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1400/2797] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:13:59.845008 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b4a29-c286-4dcd-8651-b3eed84c5177 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:13:59.855152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b4a29-c286-4dcd-8651-b3eed84c5177 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:13:59.859655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcd042-7f7f-4b51-b192-00b746731948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:13:59.860558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcd042-7f7f-4b51-b192-00b746731948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.860839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcd042-7f7f-4b51-b192-00b746731948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.861113 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcd042-7f7f-4b51-b192-00b746731948 admin admin] PATH_INFO: `/v3/projects/801d840f4b2940dc90067c09c271aa78/users/e1d3c8a88186413c89c3ec44b9ecb17c/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.863052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcd042-7f7f-4b51-b192-00b746731948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801d840f4b2940dc90067c09c271aa78, user_id=e1d3c8a88186413c89c3ec44b9ecb17c, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:13:59.865685 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcd042-7f7f-4b51-b192-00b746731948 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:13:59.875723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1398/2798] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:13:59 2022] PUT /identity/v3/projects/801d840f4b2940dc90067c09c271aa78/users/e1d3c8a88186413c89c3ec44b9ecb17c/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:13:59.888485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258cbdf-ba21-4a31-938d-80941d0f3d2d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:13:59.888776 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258cbdf-ba21-4a31-938d-80941d0f3d2d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:13:59.889054 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258cbdf-ba21-4a31-938d-80941d0f3d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:13:59.917713 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3258cbdf-ba21-4a31-938d-80941d0f3d2d None None] MFA Rules not processed for user `e1d3c8a88186413c89c3ec44b9ecb17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:13:59.918921 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3258cbdf-ba21-4a31-938d-80941d0f3d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:13:59.999552 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1401/2799] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:13:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:00.011441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcd042-7f7f-4b51-b192-00b746731948 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:00.013182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a5fcd042-7f7f-4b51-b192-00b746731948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:00.069996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcd042-7f7f-4b51-b192-00b746731948 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:00.074282 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07829fbe-f0ab-4fac-b1cf-032c2cc7dced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:00.074930 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829fbe-f0ab-4fac-b1cf-032c2cc7dced service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:00.075198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829fbe-f0ab-4fac-b1cf-032c2cc7dced service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:00.075474 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829fbe-f0ab-4fac-b1cf-032c2cc7dced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:00.084419 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07829fbe-f0ab-4fac-b1cf-032c2cc7dced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:00.085781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07829fbe-f0ab-4fac-b1cf-032c2cc7dced service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:00.095265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1399/2800] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:00.122782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258cbdf-ba21-4a31-938d-80941d0f3d2d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:00.163867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:00.186639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258cbdf-ba21-4a31-938d-80941d0f3d2d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:00.192217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:00.193137 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:00.193445 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:00.193726 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] PATH_INFO: `/v3/users/89d5c9de65e8454abf456bb80ff8521a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:00.194961 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89d5c9de65e8454abf456bb80ff8521a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:00.197562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:00.242319 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89d5c9de65e8454abf456bb80ff8521a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.249821 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89d5c9de65e8454abf456bb80ff8521a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.250509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89d5c9de65e8454abf456bb80ff8521a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.262914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89d5c9de65e8454abf456bb80ff8521a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.270576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89d5c9de65e8454abf456bb80ff8521a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.276644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1402/2801] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:14:00 2022] DELETE /identity/v3/users/89d5c9de65e8454abf456bb80ff8521a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:00.281238 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8311f6-b6ca-4eff-a2e3-ec64b65abac5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:00.281525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8311f6-b6ca-4eff-a2e3-ec64b65abac5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:00.281809 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8311f6-b6ca-4eff-a2e3-ec64b65abac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:00.299788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7d8311f6-b6ca-4eff-a2e3-ec64b65abac5 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:00.301005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7d8311f6-b6ca-4eff-a2e3-ec64b65abac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:00.358136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1400/2802] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:00.367429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:00.369987 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:00.427804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff157b-41dd-477e-86cb-ba9f915ed500 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:00.432216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a33e63-64ad-4042-9fae-006d36e99ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:00.432890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a33e63-64ad-4042-9fae-006d36e99ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:00.433176 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a33e63-64ad-4042-9fae-006d36e99ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:00.433455 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a33e63-64ad-4042-9fae-006d36e99ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:00.442450 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a33e63-64ad-4042-9fae-006d36e99ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:00.443883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a33e63-64ad-4042-9fae-006d36e99ad2 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:00.453624 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1403/2803] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:00.729457 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8311f6-b6ca-4eff-a2e3-ec64b65abac5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:00.730885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7d8311f6-b6ca-4eff-a2e3-ec64b65abac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:00.742737 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8311f6-b6ca-4eff-a2e3-ec64b65abac5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:00.747384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:00.748278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:00.748595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:00.748866 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] PATH_INFO: `/v3/projects/0908c2c18fb846448d82bef9a0225e73` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:00.750053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0908c2c18fb846448d82bef9a0225e73)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:00.751960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:00.769920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:00.801004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0908c2c18fb846448d82bef9a0225e73', 'request_id': 'req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.808278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0908c2c18fb846448d82bef9a0225e73', 'request_id': 'req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.814048 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0908c2c18fb846448d82bef9a0225e73', 'request_id': 'req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.825152 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] The token cache is being invalidate because project 0908c2c18fb846448d82bef9a0225e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:14:00.825440 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.826073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.827605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1401/2804] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:14:00 2022] DELETE /identity/v3/projects/0908c2c18fb846448d82bef9a0225e73 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:00.831570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a33e63-64ad-4042-9fae-006d36e99ad2 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:00.842997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a33e63-64ad-4042-9fae-006d36e99ad2 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:00.900954 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:00.901826 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:00.902111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:00.902391 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] PATH_INFO: `/v3/users/e1d3c8a88186413c89c3ec44b9ecb17c` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:00.903580 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d3c8a88186413c89c3ec44b9ecb17c)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:00.905536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:00.906986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:00.954397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d3c8a88186413c89c3ec44b9ecb17c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.961193 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d3c8a88186413c89c3ec44b9ecb17c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.961856 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d3c8a88186413c89c3ec44b9ecb17c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.974249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d3c8a88186413c89c3ec44b9ecb17c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.981941 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7dee8dbf-222a-4556-a41e-24b23120dc9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d3c8a88186413c89c3ec44b9ecb17c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:00.987820 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1404/2805] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:14:00 2022] DELETE /identity/v3/users/e1d3c8a88186413c89c3ec44b9ecb17c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:01.274598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:01.276472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:01.289948 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5905c3-5bc7-4cb5-aac5-e4f6a76e5952 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:01.347504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:01.348459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:01.348746 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:01.349025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] PATH_INFO: `/v3/projects/801d840f4b2940dc90067c09c271aa78` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:01.350210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=801d840f4b2940dc90067c09c271aa78)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:01.352112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:01.399475 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801d840f4b2940dc90067c09c271aa78', 'request_id': 'req-348822f6-23be-48a2-a99b-c3b4f37aeccc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:01.407926 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801d840f4b2940dc90067c09c271aa78', 'request_id': 'req-348822f6-23be-48a2-a99b-c3b4f37aeccc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:01.413387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801d840f4b2940dc90067c09c271aa78', 'request_id': 'req-348822f6-23be-48a2-a99b-c3b4f37aeccc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:01.425166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] The token cache is being invalidate because project 801d840f4b2940dc90067c09c271aa78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:14:01.425450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:01.426076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-348822f6-23be-48a2-a99b-c3b4f37aeccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:01.427605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1402/2806] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:14:01 2022] DELETE /identity/v3/projects/801d840f4b2940dc90067c09c271aa78 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:01.432641 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271d48c-1326-4210-9627-358da483feeb None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:01.432935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271d48c-1326-4210-9627-358da483feeb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:01.433218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271d48c-1326-4210-9627-358da483feeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:01.450506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a271d48c-1326-4210-9627-358da483feeb None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:01.451643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a271d48c-1326-4210-9627-358da483feeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:01.510791 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1405/2807] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:01.518368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44430-806c-484e-ab24-7332f6cd9731 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:01.518658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44430-806c-484e-ab24-7332f6cd9731 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:01.518936 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44430-806c-484e-ab24-7332f6cd9731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:01.537396 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ebf44430-806c-484e-ab24-7332f6cd9731 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:01.538592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ebf44430-806c-484e-ab24-7332f6cd9731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:01.547442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1403/2808] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:01.551413 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271d48c-1326-4210-9627-358da483feeb None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:01.560992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271d48c-1326-4210-9627-358da483feeb None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:01.565477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c770cd-2718-4973-9486-97d1272415bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:01.566124 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c770cd-2718-4973-9486-97d1272415bc admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:01.566402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c770cd-2718-4973-9486-97d1272415bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:01.566675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c770cd-2718-4973-9486-97d1272415bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:01.567393 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c770cd-2718-4973-9486-97d1272415bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:14:01.567718 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c770cd-2718-4973-9486-97d1272415bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:01.569351 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c770cd-2718-4973-9486-97d1272415bc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:01.580658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1406/2809] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:14:01.584920 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf44430-806c-484e-ab24-7332f6cd9731 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:01.594950 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf44430-806c-484e-ab24-7332f6cd9731 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:01.599518 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0d43-6006-4472-a09c-ecff52c268cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:01.600165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0d43-6006-4472-a09c-ecff52c268cf admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:01.600498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0d43-6006-4472-a09c-ecff52c268cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:01.600780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0d43-6006-4472-a09c-ecff52c268cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:01.601494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb0d43-6006-4472-a09c-ecff52c268cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:01.603928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb0d43-6006-4472-a09c-ecff52c268cf admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:01.629210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1404/2810] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:14:01 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:01.633153 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c770cd-2718-4973-9486-97d1272415bc admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:01.642850 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c770cd-2718-4973-9486-97d1272415bc admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:01.697741 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccc60e-2492-42e6-9814-37cdb07fe43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:01.698395 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccc60e-2492-42e6-9814-37cdb07fe43d admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:01.698676 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccc60e-2492-42e6-9814-37cdb07fe43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:01.698950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccc60e-2492-42e6-9814-37cdb07fe43d admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:01.699645 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bccc60e-2492-42e6-9814-37cdb07fe43d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:01.701637 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bccc60e-2492-42e6-9814-37cdb07fe43d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:01.710942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:01.722266 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1407/2811] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:14:01 2022] POST /identity/v3/users => generated 284 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:14:01.726182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0d43-6006-4472-a09c-ecff52c268cf admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:01.736966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0d43-6006-4472-a09c-ecff52c268cf admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:01.741502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd50b0a-368f-4b84-9158-e542a7aa5c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:01.742251 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd50b0a-368f-4b84-9158-e542a7aa5c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:01.742541 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd50b0a-368f-4b84-9158-e542a7aa5c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:01.742823 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd50b0a-368f-4b84-9158-e542a7aa5c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:01.743552 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd50b0a-368f-4b84-9158-e542a7aa5c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:01.745215 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd50b0a-368f-4b84-9158-e542a7aa5c8b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:01.746450 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:01.758169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1405/2812] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:01.762526 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccc60e-2492-42e6-9814-37cdb07fe43d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:01.772476 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccc60e-2492-42e6-9814-37cdb07fe43d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:01.778039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae787c4-9a46-4463-85c5-c7f03c722bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:01.778696 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae787c4-9a46-4463-85c5-c7f03c722bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:01.778981 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae787c4-9a46-4463-85c5-c7f03c722bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:01.779259 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae787c4-9a46-4463-85c5-c7f03c722bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:01.779978 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae787c4-9a46-4463-85c5-c7f03c722bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:01.781596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae787c4-9a46-4463-85c5-c7f03c722bb0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:01.792033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1408/2813] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:01.796435 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd50b0a-368f-4b84-9158-e542a7aa5c8b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:01.806345 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd50b0a-368f-4b84-9158-e542a7aa5c8b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:01.811089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019f477-492e-4bf6-b730-1d40a7296bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:01.811963 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f477-492e-4bf6-b730-1d40a7296bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:01.812252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f477-492e-4bf6-b730-1d40a7296bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:01.812573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f477-492e-4bf6-b730-1d40a7296bf5 admin admin] PATH_INFO: `/v3/projects/dbfc3d7e7531467e802f6480ce77a62f/users/f744e31eeb0241f48bc549df3d5ac12e/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:01.827549 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019f477-492e-4bf6-b730-1d40a7296bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbfc3d7e7531467e802f6480ce77a62f, user_id=f744e31eeb0241f48bc549df3d5ac12e, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:01.830220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019f477-492e-4bf6-b730-1d40a7296bf5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:01.843506 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1406/2814] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:01 2022] PUT /identity/v3/projects/dbfc3d7e7531467e802f6480ce77a62f/users/f744e31eeb0241f48bc549df3d5ac12e/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:01.855171 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cf8d2-baff-4b12-973c-68aeafd95a45 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:01.855631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cf8d2-baff-4b12-973c-68aeafd95a45 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:01.855734 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cf8d2-baff-4b12-973c-68aeafd95a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:01.883047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a51cf8d2-baff-4b12-973c-68aeafd95a45 None None] MFA Rules not processed for user `f744e31eeb0241f48bc549df3d5ac12e`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:01.885621 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a51cf8d2-baff-4b12-973c-68aeafd95a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:01.953942 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1409/2815] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:01.958455 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019f477-492e-4bf6-b730-1d40a7296bf5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:02.022524 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019f477-492e-4bf6-b730-1d40a7296bf5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:02.027894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa09ec-5f0c-4b25-beba-c7330fe066da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:02.028612 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa09ec-5f0c-4b25-beba-c7330fe066da admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:02.028892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa09ec-5f0c-4b25-beba-c7330fe066da admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:02.029166 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa09ec-5f0c-4b25-beba-c7330fe066da admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:02.029897 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa09ec-5f0c-4b25-beba-c7330fe066da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:02.031845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa09ec-5f0c-4b25-beba-c7330fe066da admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:02.060088 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1407/2816] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:14:01 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:02.064492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cf8d2-baff-4b12-973c-68aeafd95a45 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:02.076693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cf8d2-baff-4b12-973c-68aeafd95a45 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:02.136185 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796a8b1-8ef9-492e-b08c-3c661cc473eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:02.136995 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796a8b1-8ef9-492e-b08c-3c661cc473eb admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:02.137278 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796a8b1-8ef9-492e-b08c-3c661cc473eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:02.137555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796a8b1-8ef9-492e-b08c-3c661cc473eb admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:02.138251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796a8b1-8ef9-492e-b08c-3c661cc473eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:02.140162 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796a8b1-8ef9-492e-b08c-3c661cc473eb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:02.158713 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1410/2817] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:14:02 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:14:02.162730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa09ec-5f0c-4b25-beba-c7330fe066da admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:02.173894 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa09ec-5f0c-4b25-beba-c7330fe066da admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:02.178544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44804b1-f43a-4dab-9210-739c58246c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:02.179216 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44804b1-f43a-4dab-9210-739c58246c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:02.179498 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44804b1-f43a-4dab-9210-739c58246c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:02.179773 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44804b1-f43a-4dab-9210-739c58246c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:02.180537 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44804b1-f43a-4dab-9210-739c58246c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:02.182690 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44804b1-f43a-4dab-9210-739c58246c35 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:02.193442 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1408/2818] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:02.198797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796a8b1-8ef9-492e-b08c-3c661cc473eb admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:02.208887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796a8b1-8ef9-492e-b08c-3c661cc473eb admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:02.213971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61386a06-1d18-414b-8bcb-2c85cc315246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:02.214890 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61386a06-1d18-414b-8bcb-2c85cc315246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:02.215173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61386a06-1d18-414b-8bcb-2c85cc315246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:02.215451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61386a06-1d18-414b-8bcb-2c85cc315246 admin admin] PATH_INFO: `/v3/projects/2feb8c1eb1eb43d3916d89b35771e941/users/a90ff344fb854e309fa459eccd149651/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:02.230353 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61386a06-1d18-414b-8bcb-2c85cc315246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2feb8c1eb1eb43d3916d89b35771e941, user_id=a90ff344fb854e309fa459eccd149651, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:02.233090 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61386a06-1d18-414b-8bcb-2c85cc315246 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:02.246554 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1411/2819] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:02 2022] PUT /identity/v3/projects/2feb8c1eb1eb43d3916d89b35771e941/users/a90ff344fb854e309fa459eccd149651/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:02.250421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44804b1-f43a-4dab-9210-739c58246c35 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:02.261357 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44804b1-f43a-4dab-9210-739c58246c35 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:02.318405 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1192b0a-7786-42e9-9734-6024f28d9dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:02.319070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1192b0a-7786-42e9-9734-6024f28d9dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:02.319335 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1192b0a-7786-42e9-9734-6024f28d9dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:02.319614 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1192b0a-7786-42e9-9734-6024f28d9dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:02.320353 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1192b0a-7786-42e9-9734-6024f28d9dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:02.322525 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1192b0a-7786-42e9-9734-6024f28d9dc7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:02.332258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1409/2820] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:02.336549 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61386a06-1d18-414b-8bcb-2c85cc315246 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:02.346597 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61386a06-1d18-414b-8bcb-2c85cc315246 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:02.351111 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7900b22-d1ae-4412-b9df-b8ed016300e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:02.351978 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7900b22-d1ae-4412-b9df-b8ed016300e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:02.352256 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7900b22-d1ae-4412-b9df-b8ed016300e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:02.352562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7900b22-d1ae-4412-b9df-b8ed016300e1 admin admin] PATH_INFO: `/v3/projects/2feb8c1eb1eb43d3916d89b35771e941/users/a90ff344fb854e309fa459eccd149651/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:02.354508 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7900b22-d1ae-4412-b9df-b8ed016300e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2feb8c1eb1eb43d3916d89b35771e941, user_id=a90ff344fb854e309fa459eccd149651, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:02.357143 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7900b22-d1ae-4412-b9df-b8ed016300e1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:02.367199 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1412/2821] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:02 2022] PUT /identity/v3/projects/2feb8c1eb1eb43d3916d89b35771e941/users/a90ff344fb854e309fa459eccd149651/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:02.378013 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af600cb-70d1-490b-a989-e39ee7803f40 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:02.378301 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af600cb-70d1-490b-a989-e39ee7803f40 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:02.378585 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af600cb-70d1-490b-a989-e39ee7803f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:02.404622 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1af600cb-70d1-490b-a989-e39ee7803f40 None None] MFA Rules not processed for user `a90ff344fb854e309fa459eccd149651`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:02.405822 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1af600cb-70d1-490b-a989-e39ee7803f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:02.435424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:02.474005 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1410/2822] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:02.485179 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7900b22-d1ae-4412-b9df-b8ed016300e1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:02.486903 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c7900b22-d1ae-4412-b9df-b8ed016300e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:02.518653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:02.546363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7900b22-d1ae-4412-b9df-b8ed016300e1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:02.549993 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87378b-2591-4c45-9a6d-b0433e3a3fc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:02.550631 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87378b-2591-4c45-9a6d-b0433e3a3fc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:02.550915 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87378b-2591-4c45-9a6d-b0433e3a3fc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:02.551195 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87378b-2591-4c45-9a6d-b0433e3a3fc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:02.560806 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87378b-2591-4c45-9a6d-b0433e3a3fc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:02.562239 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87378b-2591-4c45-9a6d-b0433e3a3fc8 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:02.571482 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1413/2823] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:02.647307 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af600cb-70d1-490b-a989-e39ee7803f40 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:02.657124 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af600cb-70d1-490b-a989-e39ee7803f40 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:02.660726 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d357b-6c4d-4b8a-999e-1ab4fd4c49fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:02.661360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d357b-6c4d-4b8a-999e-1ab4fd4c49fa service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:02.661642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d357b-6c4d-4b8a-999e-1ab4fd4c49fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:02.661922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d357b-6c4d-4b8a-999e-1ab4fd4c49fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:02.719292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d357b-6c4d-4b8a-999e-1ab4fd4c49fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:02.720773 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d357b-6c4d-4b8a-999e-1ab4fd4c49fa service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:02.730819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1411/2824] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:07.364359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87378b-2591-4c45-9a6d-b0433e3a3fc8 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:07.427186 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87378b-2591-4c45-9a6d-b0433e3a3fc8 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:07.432594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:07.433512 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:07.433815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:07.434089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] PATH_INFO: `/v3/users/f744e31eeb0241f48bc549df3d5ac12e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:07.435268 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f744e31eeb0241f48bc549df3d5ac12e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:07.437222 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:07.483070 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f744e31eeb0241f48bc549df3d5ac12e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:07.489665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f744e31eeb0241f48bc549df3d5ac12e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:07.490321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f744e31eeb0241f48bc549df3d5ac12e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:07.502158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f744e31eeb0241f48bc549df3d5ac12e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:07.509288 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f744e31eeb0241f48bc549df3d5ac12e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:07.516462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1414/2825] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:14:07 2022] DELETE /identity/v3/users/f744e31eeb0241f48bc549df3d5ac12e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:07.521127 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f1471-cb23-4a75-af68-2909a6506d8b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:07.521622 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f1471-cb23-4a75-af68-2909a6506d8b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:07.521722 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f1471-cb23-4a75-af68-2909a6506d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:07.538721 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-574f1471-cb23-4a75-af68-2909a6506d8b None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:07.539929 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-574f1471-cb23-4a75-af68-2909a6506d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:07.598417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1412/2826] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:07.608430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:07.609862 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:07.663225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:07.667515 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820c6df-158c-40ee-9e8e-b100bb0387bd admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:07.672838 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b9b9c-d629-4dc9-bff7-f7caaa6bbca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:07.673604 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b9b9c-d629-4dc9-bff7-f7caaa6bbca8 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:07.674033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b9b9c-d629-4dc9-bff7-f7caaa6bbca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:07.674562 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b9b9c-d629-4dc9-bff7-f7caaa6bbca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:07.684540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b9b9c-d629-4dc9-bff7-f7caaa6bbca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:07.685956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b9b9c-d629-4dc9-bff7-f7caaa6bbca8 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:07.695219 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1415/2827] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:07.960388 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f1471-cb23-4a75-af68-2909a6506d8b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:07.961805 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-574f1471-cb23-4a75-af68-2909a6506d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:07.973657 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f1471-cb23-4a75-af68-2909a6506d8b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:07.979119 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:07.980162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:07.980476 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:07.980752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] PATH_INFO: `/v3/projects/dbfc3d7e7531467e802f6480ce77a62f` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:07.981941 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbfc3d7e7531467e802f6480ce77a62f)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:07.983836 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:08.035652 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbfc3d7e7531467e802f6480ce77a62f', 'request_id': 'req-ac725ccc-6ef2-4712-a220-2b69185172fc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.044263 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbfc3d7e7531467e802f6480ce77a62f', 'request_id': 'req-ac725ccc-6ef2-4712-a220-2b69185172fc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.048511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:08.050821 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbfc3d7e7531467e802f6480ce77a62f', 'request_id': 'req-ac725ccc-6ef2-4712-a220-2b69185172fc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.064620 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] The token cache is being invalidate because project dbfc3d7e7531467e802f6480ce77a62f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:14:08.064904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.065527 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.067086 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1413/2828] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:14:07 2022] DELETE /identity/v3/projects/dbfc3d7e7531467e802f6480ce77a62f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:08.070979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b9b9c-d629-4dc9-bff7-f7caaa6bbca8 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:08.080977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b9b9c-d629-4dc9-bff7-f7caaa6bbca8 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:08.136403 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:08.137271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:08.137553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:08.137828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] PATH_INFO: `/v3/users/a90ff344fb854e309fa459eccd149651` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:08.138994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a90ff344fb854e309fa459eccd149651)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:08.140953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:08.186269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a90ff344fb854e309fa459eccd149651'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.192984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a90ff344fb854e309fa459eccd149651'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.193639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a90ff344fb854e309fa459eccd149651'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.205249 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a90ff344fb854e309fa459eccd149651'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.213651 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-991bcc49-059a-4653-ac13-bb9aad91ac06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a90ff344fb854e309fa459eccd149651'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.219867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1416/2829] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:14:08 2022] DELETE /identity/v3/users/a90ff344fb854e309fa459eccd149651 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:08.485341 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:08.486754 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:08.500820 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac725ccc-6ef2-4712-a220-2b69185172fc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:08.557838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:08.558698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:08.558982 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:08.559259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] PATH_INFO: `/v3/projects/2feb8c1eb1eb43d3916d89b35771e941` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:08.560502 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2feb8c1eb1eb43d3916d89b35771e941)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:08.562390 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:08.607630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2feb8c1eb1eb43d3916d89b35771e941', 'request_id': 'req-fba16233-63b7-45c1-808d-09ecc8cdb25c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.614889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2feb8c1eb1eb43d3916d89b35771e941', 'request_id': 'req-fba16233-63b7-45c1-808d-09ecc8cdb25c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.621380 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2feb8c1eb1eb43d3916d89b35771e941', 'request_id': 'req-fba16233-63b7-45c1-808d-09ecc8cdb25c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.631979 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] The token cache is being invalidate because project 2feb8c1eb1eb43d3916d89b35771e941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:14:08.632269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.632918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fba16233-63b7-45c1-808d-09ecc8cdb25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:08.634428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1414/2830] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:14:08 2022] DELETE /identity/v3/projects/2feb8c1eb1eb43d3916d89b35771e941 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:08.639843 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c4924-780c-486a-be49-715cca34fbce None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:08.640370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c4924-780c-486a-be49-715cca34fbce None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:08.640464 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c4924-780c-486a-be49-715cca34fbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:08.657584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ca3c4924-780c-486a-be49-715cca34fbce None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:08.658767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ca3c4924-780c-486a-be49-715cca34fbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:08.719506 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1417/2831] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:08.727273 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a166e-5714-4461-925e-ff6026c5553d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:08.727567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a166e-5714-4461-925e-ff6026c5553d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:08.727853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a166e-5714-4461-925e-ff6026c5553d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:08.746850 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a60a166e-5714-4461-925e-ff6026c5553d None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:08.748053 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a60a166e-5714-4461-925e-ff6026c5553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:08.755966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1415/2832] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:08.760055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c4924-780c-486a-be49-715cca34fbce None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:08.769789 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c4924-780c-486a-be49-715cca34fbce None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:08.774293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65431d-1c24-4ba3-9571-5bbb0ba1868b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:08.774939 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65431d-1c24-4ba3-9571-5bbb0ba1868b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:08.775221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65431d-1c24-4ba3-9571-5bbb0ba1868b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:08.775578 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65431d-1c24-4ba3-9571-5bbb0ba1868b admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:08.776358 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65431d-1c24-4ba3-9571-5bbb0ba1868b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:14:08.776671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65431d-1c24-4ba3-9571-5bbb0ba1868b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:08.778267 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65431d-1c24-4ba3-9571-5bbb0ba1868b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:08.785006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:08.794570 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1418/2833] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 18:14:08.795955 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a166e-5714-4461-925e-ff6026c5553d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:08.805097 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a166e-5714-4461-925e-ff6026c5553d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:08.811246 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a44d4-79ec-4b1e-8fea-9a2fc5850368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:08.811913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a44d4-79ec-4b1e-8fea-9a2fc5850368 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:08.812222 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a44d4-79ec-4b1e-8fea-9a2fc5850368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:08.812539 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a44d4-79ec-4b1e-8fea-9a2fc5850368 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:08.813266 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a44d4-79ec-4b1e-8fea-9a2fc5850368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:08.815187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a44d4-79ec-4b1e-8fea-9a2fc5850368 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:08.839976 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1416/2834] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:14:08 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:08.843974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65431d-1c24-4ba3-9571-5bbb0ba1868b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:08.854607 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65431d-1c24-4ba3-9571-5bbb0ba1868b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:08.911887 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e7c8a-122b-42b0-a4a2-2f974aa4b9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:08.912577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e7c8a-122b-42b0-a4a2-2f974aa4b9dc admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:08.912864 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e7c8a-122b-42b0-a4a2-2f974aa4b9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:08.913163 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e7c8a-122b-42b0-a4a2-2f974aa4b9dc admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:08.913853 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e7c8a-122b-42b0-a4a2-2f974aa4b9dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:08.915779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e7c8a-122b-42b0-a4a2-2f974aa4b9dc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:08.936673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1419/2835] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:14:08 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:14:08.940740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a44d4-79ec-4b1e-8fea-9a2fc5850368 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:08.951032 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a44d4-79ec-4b1e-8fea-9a2fc5850368 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:08.956080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6a47a-b4c4-4b16-af3e-76ad13bcbb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:08.956785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6a47a-b4c4-4b16-af3e-76ad13bcbb3a admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:08.957073 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6a47a-b4c4-4b16-af3e-76ad13bcbb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:08.957354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6a47a-b4c4-4b16-af3e-76ad13bcbb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:08.958087 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6a47a-b4c4-4b16-af3e-76ad13bcbb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:08.959691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6a47a-b4c4-4b16-af3e-76ad13bcbb3a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:08.971025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1417/2836] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:08.974755 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e7c8a-122b-42b0-a4a2-2f974aa4b9dc admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:08.985877 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e7c8a-122b-42b0-a4a2-2f974aa4b9dc admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:08.990411 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120a2cd-86f3-496c-a07a-d5745a4f3b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:08.992422 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a2cd-86f3-496c-a07a-d5745a4f3b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:08.992711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a2cd-86f3-496c-a07a-d5745a4f3b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:08.992986 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a2cd-86f3-496c-a07a-d5745a4f3b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:08.993712 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120a2cd-86f3-496c-a07a-d5745a4f3b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:08.995555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120a2cd-86f3-496c-a07a-d5745a4f3b18 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:09.006380 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1420/2837] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:09.011037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6a47a-b4c4-4b16-af3e-76ad13bcbb3a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:09.021159 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6a47a-b4c4-4b16-af3e-76ad13bcbb3a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:09.026397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a16b0-6c03-4b80-abf0-3dc7625d25ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:09.027278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a16b0-6c03-4b80-abf0-3dc7625d25ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:09.027566 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a16b0-6c03-4b80-abf0-3dc7625d25ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:09.027842 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a16b0-6c03-4b80-abf0-3dc7625d25ea admin admin] PATH_INFO: `/v3/projects/94ddc6f181d04d10b4f2f8035dfbcf80/users/6cbb84238a5f4b0386f09965baec8b0b/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:09.043169 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a16b0-6c03-4b80-abf0-3dc7625d25ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ddc6f181d04d10b4f2f8035dfbcf80, user_id=6cbb84238a5f4b0386f09965baec8b0b, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:09.046198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a16b0-6c03-4b80-abf0-3dc7625d25ea admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:09.058922 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1418/2838] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:09 2022] PUT /identity/v3/projects/94ddc6f181d04d10b4f2f8035dfbcf80/users/6cbb84238a5f4b0386f09965baec8b0b/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:09.069969 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8c69b-d2f2-45af-8a6c-b831a78872b0 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:09.070260 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8c69b-d2f2-45af-8a6c-b831a78872b0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:09.070538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8c69b-d2f2-45af-8a6c-b831a78872b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:09.097509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a7b8c69b-d2f2-45af-8a6c-b831a78872b0 None None] MFA Rules not processed for user `6cbb84238a5f4b0386f09965baec8b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:09.098717 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a7b8c69b-d2f2-45af-8a6c-b831a78872b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:09.161237 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1421/2839] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:14:09.173198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a16b0-6c03-4b80-abf0-3dc7625d25ea admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:09.174904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ea7a16b0-6c03-4b80-abf0-3dc7625d25ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:09.193544 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:09.234787 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a16b0-6c03-4b80-abf0-3dc7625d25ea admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:09.238471 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed6e90-5c9f-4d8f-ba47-c01bb292f975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:09.239112 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed6e90-5c9f-4d8f-ba47-c01bb292f975 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:09.239399 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed6e90-5c9f-4d8f-ba47-c01bb292f975 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:09.239676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed6e90-5c9f-4d8f-ba47-c01bb292f975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:09.248417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed6e90-5c9f-4d8f-ba47-c01bb292f975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:09.249848 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed6e90-5c9f-4d8f-ba47-c01bb292f975 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:09.258507 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1419/2840] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:17.784900 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8c69b-d2f2-45af-8a6c-b831a78872b0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:17.848354 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8c69b-d2f2-45af-8a6c-b831a78872b0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:17.852813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:17.853683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:17.853964 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:17.854240 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] PATH_INFO: `/v3/users/6cbb84238a5f4b0386f09965baec8b0b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:17.855417 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cbb84238a5f4b0386f09965baec8b0b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:17.857633 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:17.862932 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:17.906005 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cbb84238a5f4b0386f09965baec8b0b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:17.912789 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cbb84238a5f4b0386f09965baec8b0b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:17.913458 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cbb84238a5f4b0386f09965baec8b0b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:17.926173 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cbb84238a5f4b0386f09965baec8b0b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:17.935532 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cbb84238a5f4b0386f09965baec8b0b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:17.942001 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1422/2841] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:14:17 2022] DELETE /identity/v3/users/6cbb84238a5f4b0386f09965baec8b0b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:17.946463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31118715-b149-464b-8b01-e96b1e19856f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:17.946871 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31118715-b149-464b-8b01-e96b1e19856f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:17.947071 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31118715-b149-464b-8b01-e96b1e19856f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:17.964158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-31118715-b149-464b-8b01-e96b1e19856f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:17.965366 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-31118715-b149-464b-8b01-e96b1e19856f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:18.026935 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1420/2842] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:18.037184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:18.038606 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:18.096522 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfa916-873d-482b-9c75-4256c7c7f96a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:18.100235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490fa32-f0f5-400f-9ae9-6c48a01fedea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:18.100907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490fa32-f0f5-400f-9ae9-6c48a01fedea service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.101188 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490fa32-f0f5-400f-9ae9-6c48a01fedea service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.101462 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490fa32-f0f5-400f-9ae9-6c48a01fedea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.110695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490fa32-f0f5-400f-9ae9-6c48a01fedea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:18.112120 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490fa32-f0f5-400f-9ae9-6c48a01fedea service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:18.121472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1423/2843] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:18.393794 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31118715-b149-464b-8b01-e96b1e19856f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:18.395245 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-31118715-b149-464b-8b01-e96b1e19856f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:18.409701 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31118715-b149-464b-8b01-e96b1e19856f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:18.415460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:18.416218 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.416536 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.416810 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] PATH_INFO: `/v3/projects/94ddc6f181d04d10b4f2f8035dfbcf80` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.418208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ddc6f181d04d10b4f2f8035dfbcf80)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:18.420970 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:18.466838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ddc6f181d04d10b4f2f8035dfbcf80', 'request_id': 'req-a30f1107-bd7c-4c77-a23b-43cff4590e70'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:18.473734 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ddc6f181d04d10b4f2f8035dfbcf80', 'request_id': 'req-a30f1107-bd7c-4c77-a23b-43cff4590e70'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:18.479137 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ddc6f181d04d10b4f2f8035dfbcf80', 'request_id': 'req-a30f1107-bd7c-4c77-a23b-43cff4590e70'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:18.490755 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] The token cache is being invalidate because project 94ddc6f181d04d10b4f2f8035dfbcf80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:14:18.491036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:18.491660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a30f1107-bd7c-4c77-a23b-43cff4590e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:18.493208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1421/2844] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:14:18 2022] DELETE /identity/v3/projects/94ddc6f181d04d10b4f2f8035dfbcf80 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:18.498599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdd567-4f59-4366-93e6-f0d8bbc06fce None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.498889 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdd567-4f59-4366-93e6-f0d8bbc06fce None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.499191 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdd567-4f59-4366-93e6-f0d8bbc06fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.516858 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6efdd567-4f59-4366-93e6-f0d8bbc06fce None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:18.518059 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6efdd567-4f59-4366-93e6-f0d8bbc06fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:18.576015 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1424/2845] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:18.583648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24125ffa-a8c6-4c2e-9efa-54956dc8b41f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.583919 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24125ffa-a8c6-4c2e-9efa-54956dc8b41f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.584205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24125ffa-a8c6-4c2e-9efa-54956dc8b41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.602520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-24125ffa-a8c6-4c2e-9efa-54956dc8b41f None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:18.603703 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-24125ffa-a8c6-4c2e-9efa-54956dc8b41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:18.612165 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1422/2846] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:18.616107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdd567-4f59-4366-93e6-f0d8bbc06fce None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:18.625829 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdd567-4f59-4366-93e6-f0d8bbc06fce None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:18.630375 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a8811-bd30-4bb6-b5ba-33997473dd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:18.631024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a8811-bd30-4bb6-b5ba-33997473dd03 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.631309 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a8811-bd30-4bb6-b5ba-33997473dd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.631584 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a8811-bd30-4bb6-b5ba-33997473dd03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.632325 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a8811-bd30-4bb6-b5ba-33997473dd03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:14:18.632638 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a8811-bd30-4bb6-b5ba-33997473dd03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:18.634225 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a8811-bd30-4bb6-b5ba-33997473dd03 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:18.645746 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1425/2847] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:14:18.649885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24125ffa-a8c6-4c2e-9efa-54956dc8b41f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:18.659543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24125ffa-a8c6-4c2e-9efa-54956dc8b41f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:18.664078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdca93b-5b44-4db2-b624-4be2d2c968fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:18.664768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdca93b-5b44-4db2-b624-4be2d2c968fb admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.665045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdca93b-5b44-4db2-b624-4be2d2c968fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.665317 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdca93b-5b44-4db2-b624-4be2d2c968fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.665992 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdca93b-5b44-4db2-b624-4be2d2c968fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:18.667845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdca93b-5b44-4db2-b624-4be2d2c968fb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:18.694172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1423/2848] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:14:18 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:14:18.698295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a8811-bd30-4bb6-b5ba-33997473dd03 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:18.708101 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a8811-bd30-4bb6-b5ba-33997473dd03 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:18.720345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:18.766467 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5a5ae-9f66-480b-8d39-8239fe82d6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:18.767128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5a5ae-9f66-480b-8d39-8239fe82d6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.767410 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5a5ae-9f66-480b-8d39-8239fe82d6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.767685 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5a5ae-9f66-480b-8d39-8239fe82d6d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.768431 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5a5ae-9f66-480b-8d39-8239fe82d6d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:18.770347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5a5ae-9f66-480b-8d39-8239fe82d6d6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:18.789884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1426/2849] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:14:18 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:18.793947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdca93b-5b44-4db2-b624-4be2d2c968fb admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:18.804674 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdca93b-5b44-4db2-b624-4be2d2c968fb admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:18.809226 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d45a45-7faf-43f6-a6d9-f589dde0843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:18.809929 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d45a45-7faf-43f6-a6d9-f589dde0843c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.810447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d45a45-7faf-43f6-a6d9-f589dde0843c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.810447 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d45a45-7faf-43f6-a6d9-f589dde0843c admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.811191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d45a45-7faf-43f6-a6d9-f589dde0843c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:18.812814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d45a45-7faf-43f6-a6d9-f589dde0843c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:18.822960 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1424/2850] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:18.826834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5a5ae-9f66-480b-8d39-8239fe82d6d6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:18.836690 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5a5ae-9f66-480b-8d39-8239fe82d6d6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:18.842280 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6084ae7-ca75-4d77-b099-e96da050eda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:18.842937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6084ae7-ca75-4d77-b099-e96da050eda5 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.843221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6084ae7-ca75-4d77-b099-e96da050eda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.843498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6084ae7-ca75-4d77-b099-e96da050eda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.844221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6084ae7-ca75-4d77-b099-e96da050eda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:18.845841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6084ae7-ca75-4d77-b099-e96da050eda5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:18.855587 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1427/2851] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:18.860162 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d45a45-7faf-43f6-a6d9-f589dde0843c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:18.865261 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:18.871480 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d45a45-7faf-43f6-a6d9-f589dde0843c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:18.877227 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fb55a-5659-4912-8db1-103f80da9b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:18.878094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fb55a-5659-4912-8db1-103f80da9b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.878372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fb55a-5659-4912-8db1-103f80da9b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.878643 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fb55a-5659-4912-8db1-103f80da9b33 admin admin] PATH_INFO: `/v3/projects/04c8dea3ca3a4c679c98997b6beff7cf/users/992423d4fe5240aca76d51a15a7d6f01/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.893885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fb55a-5659-4912-8db1-103f80da9b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c8dea3ca3a4c679c98997b6beff7cf, user_id=992423d4fe5240aca76d51a15a7d6f01, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:18.896551 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fb55a-5659-4912-8db1-103f80da9b33 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:18.909400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1425/2852] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:18 2022] PUT /identity/v3/projects/04c8dea3ca3a4c679c98997b6beff7cf/users/992423d4fe5240aca76d51a15a7d6f01/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:18.920389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58001e83-60ed-42ec-9fe7-c7359ae11f0b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:18.920686 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58001e83-60ed-42ec-9fe7-c7359ae11f0b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:18.920971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58001e83-60ed-42ec-9fe7-c7359ae11f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:18.948451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-58001e83-60ed-42ec-9fe7-c7359ae11f0b None None] MFA Rules not processed for user `992423d4fe5240aca76d51a15a7d6f01`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:18.949664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-58001e83-60ed-42ec-9fe7-c7359ae11f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:19.014917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1428/2853] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:19.019348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fb55a-5659-4912-8db1-103f80da9b33 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.080642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fb55a-5659-4912-8db1-103f80da9b33 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.085191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c5e7-c6e1-482b-b836-59956399765a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.085844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c5e7-c6e1-482b-b836-59956399765a admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.086125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c5e7-c6e1-482b-b836-59956399765a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.086400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c5e7-c6e1-482b-b836-59956399765a admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.087099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29c5e7-c6e1-482b-b836-59956399765a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.089019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29c5e7-c6e1-482b-b836-59956399765a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.114874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1426/2854] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:14:19 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:19.118943 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58001e83-60ed-42ec-9fe7-c7359ae11f0b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.129623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58001e83-60ed-42ec-9fe7-c7359ae11f0b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.185646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888394e-f479-499e-a071-a58383791834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.186315 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888394e-f479-499e-a071-a58383791834 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.186599 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888394e-f479-499e-a071-a58383791834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.186875 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888394e-f479-499e-a071-a58383791834 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.187553 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888394e-f479-499e-a071-a58383791834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.189492 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888394e-f479-499e-a071-a58383791834 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.209500 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1429/2855] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:14:19 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:14:19.213421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c5e7-c6e1-482b-b836-59956399765a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.224259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c5e7-c6e1-482b-b836-59956399765a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.228851 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23013503-23f1-4fb3-92cf-a78dca657c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.229504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23013503-23f1-4fb3-92cf-a78dca657c78 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.229788 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23013503-23f1-4fb3-92cf-a78dca657c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.230070 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23013503-23f1-4fb3-92cf-a78dca657c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.230795 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23013503-23f1-4fb3-92cf-a78dca657c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.232664 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23013503-23f1-4fb3-92cf-a78dca657c78 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.243030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1427/2856] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:19.246884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888394e-f479-499e-a071-a58383791834 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.257955 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888394e-f479-499e-a071-a58383791834 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.262652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb395af3-a6d9-4bb1-8740-d47ecbf4fb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.263844 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb395af3-a6d9-4bb1-8740-d47ecbf4fb1d admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.264322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb395af3-a6d9-4bb1-8740-d47ecbf4fb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.264649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb395af3-a6d9-4bb1-8740-d47ecbf4fb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.265381 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb395af3-a6d9-4bb1-8740-d47ecbf4fb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.267009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb395af3-a6d9-4bb1-8740-d47ecbf4fb1d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.276905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1430/2857] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:19.281509 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23013503-23f1-4fb3-92cf-a78dca657c78 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.291663 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23013503-23f1-4fb3-92cf-a78dca657c78 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.296940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f4990-8ae3-471a-ba23-1c018af666cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.297994 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f4990-8ae3-471a-ba23-1c018af666cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.298174 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f4990-8ae3-471a-ba23-1c018af666cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.298454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f4990-8ae3-471a-ba23-1c018af666cc admin admin] PATH_INFO: `/v3/projects/0b5c2ee8ca0841e2a26efff9fab20162/users/163bca3c3f4f4c28ae79ae1090ec7ef8/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.314104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f4990-8ae3-471a-ba23-1c018af666cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5c2ee8ca0841e2a26efff9fab20162, user_id=163bca3c3f4f4c28ae79ae1090ec7ef8, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.316804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f4990-8ae3-471a-ba23-1c018af666cc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.329897 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1428/2858] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:19 2022] PUT /identity/v3/projects/0b5c2ee8ca0841e2a26efff9fab20162/users/163bca3c3f4f4c28ae79ae1090ec7ef8/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:14:19.341271 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03217ea2-8a18-40d3-986c-15bfd05e7da1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.341550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03217ea2-8a18-40d3-986c-15bfd05e7da1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.341870 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03217ea2-8a18-40d3-986c-15bfd05e7da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.347675 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:19.369481 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-03217ea2-8a18-40d3-986c-15bfd05e7da1 None None] MFA Rules not processed for user `163bca3c3f4f4c28ae79ae1090ec7ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:19.371115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-03217ea2-8a18-40d3-986c-15bfd05e7da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:19.435731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1431/2859] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:19.440313 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f4990-8ae3-471a-ba23-1c018af666cc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.502010 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f4990-8ae3-471a-ba23-1c018af666cc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.506733 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac269689-b9ca-4640-98f3-509c9e429b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.507455 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac269689-b9ca-4640-98f3-509c9e429b2f admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.507739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac269689-b9ca-4640-98f3-509c9e429b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.508015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac269689-b9ca-4640-98f3-509c9e429b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.508768 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac269689-b9ca-4640-98f3-509c9e429b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.510668 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac269689-b9ca-4640-98f3-509c9e429b2f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.536037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1429/2860] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:14:19 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:19.540130 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03217ea2-8a18-40d3-986c-15bfd05e7da1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.551123 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03217ea2-8a18-40d3-986c-15bfd05e7da1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.608615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6b981-cb70-4d4a-8ee0-7878cdd4dbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.609205 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6b981-cb70-4d4a-8ee0-7878cdd4dbaf admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.609485 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6b981-cb70-4d4a-8ee0-7878cdd4dbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.609758 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6b981-cb70-4d4a-8ee0-7878cdd4dbaf admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.610448 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6b981-cb70-4d4a-8ee0-7878cdd4dbaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.612376 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6b981-cb70-4d4a-8ee0-7878cdd4dbaf admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.631731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1432/2861] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:14:19 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:19.635761 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac269689-b9ca-4640-98f3-509c9e429b2f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.646771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac269689-b9ca-4640-98f3-509c9e429b2f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.651377 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7f402-701d-43cd-a66f-e81b0093c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.652044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7f402-701d-43cd-a66f-e81b0093c8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.652354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7f402-701d-43cd-a66f-e81b0093c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.652635 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7f402-701d-43cd-a66f-e81b0093c8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.653362 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7f402-701d-43cd-a66f-e81b0093c8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.654936 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7f402-701d-43cd-a66f-e81b0093c8b6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.665387 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1430/2862] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:19.670040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6b981-cb70-4d4a-8ee0-7878cdd4dbaf admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.679656 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6b981-cb70-4d4a-8ee0-7878cdd4dbaf admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.684148 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce53508-3593-4597-aa8a-5f0881b952a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.685694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce53508-3593-4597-aa8a-5f0881b952a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.685974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce53508-3593-4597-aa8a-5f0881b952a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.686247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce53508-3593-4597-aa8a-5f0881b952a3 admin admin] PATH_INFO: `/v3/projects/8142c7421b1f436eb720685bb0733815/users/6e4fbdceec91417a950303237516c70a/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.700436 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce53508-3593-4597-aa8a-5f0881b952a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8142c7421b1f436eb720685bb0733815, user_id=6e4fbdceec91417a950303237516c70a, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.703282 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce53508-3593-4597-aa8a-5f0881b952a3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.715343 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1433/2863] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:19 2022] PUT /identity/v3/projects/8142c7421b1f436eb720685bb0733815/users/6e4fbdceec91417a950303237516c70a/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:19.719221 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7f402-701d-43cd-a66f-e81b0093c8b6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.724413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:19.730600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7f402-701d-43cd-a66f-e81b0093c8b6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.789062 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f209aa-e015-4a5c-b72d-60eed414a93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.789720 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f209aa-e015-4a5c-b72d-60eed414a93d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.790000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f209aa-e015-4a5c-b72d-60eed414a93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.790276 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f209aa-e015-4a5c-b72d-60eed414a93d admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.791000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f209aa-e015-4a5c-b72d-60eed414a93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.792601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f209aa-e015-4a5c-b72d-60eed414a93d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.802901 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1431/2864] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:19.807201 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce53508-3593-4597-aa8a-5f0881b952a3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.818204 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce53508-3593-4597-aa8a-5f0881b952a3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:19.822883 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31724b5b-bbf3-49e7-a088-1c891e686a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:19.823759 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31724b5b-bbf3-49e7-a088-1c891e686a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.824044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31724b5b-bbf3-49e7-a088-1c891e686a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.824356 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31724b5b-bbf3-49e7-a088-1c891e686a03 admin admin] PATH_INFO: `/v3/projects/8142c7421b1f436eb720685bb0733815/users/6e4fbdceec91417a950303237516c70a/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.826472 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31724b5b-bbf3-49e7-a088-1c891e686a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8142c7421b1f436eb720685bb0733815, user_id=6e4fbdceec91417a950303237516c70a, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:19.829292 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31724b5b-bbf3-49e7-a088-1c891e686a03 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:19.839384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1434/2865] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:19 2022] PUT /identity/v3/projects/8142c7421b1f436eb720685bb0733815/users/6e4fbdceec91417a950303237516c70a/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:19.849834 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c767c5-d6b0-44ec-91a6-e9bf55ff5366 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:19.850125 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c767c5-d6b0-44ec-91a6-e9bf55ff5366 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:19.850402 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c767c5-d6b0-44ec-91a6-e9bf55ff5366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:19.877179 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-81c767c5-d6b0-44ec-91a6-e9bf55ff5366 None None] MFA Rules not processed for user `6e4fbdceec91417a950303237516c70a`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:19.878360 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-81c767c5-d6b0-44ec-91a6-e9bf55ff5366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:19.948678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1432/2866] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 18:14:19.959891 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31724b5b-bbf3-49e7-a088-1c891e686a03 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:19.961622 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-31724b5b-bbf3-49e7-a088-1c891e686a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:20.009542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:20.021050 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31724b5b-bbf3-49e7-a088-1c891e686a03 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:20.024735 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8394e5-25f9-459f-b8ec-311b3385bc3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:20.025371 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8394e5-25f9-459f-b8ec-311b3385bc3e service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:20.025654 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8394e5-25f9-459f-b8ec-311b3385bc3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:20.025933 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8394e5-25f9-459f-b8ec-311b3385bc3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:20.086932 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8394e5-25f9-459f-b8ec-311b3385bc3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:20.088401 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8394e5-25f9-459f-b8ec-311b3385bc3e service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:20.098092 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1435/2867] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:21.575568 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c767c5-d6b0-44ec-91a6-e9bf55ff5366 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:21.577036 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-81c767c5-d6b0-44ec-91a6-e9bf55ff5366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:21.588686 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c767c5-d6b0-44ec-91a6-e9bf55ff5366 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:21.592108 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1c1e5-e8e3-407a-a72b-298b4cd01349 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:21.593308 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1c1e5-e8e3-407a-a72b-298b4cd01349 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:21.593592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1c1e5-e8e3-407a-a72b-298b4cd01349 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:21.593861 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1c1e5-e8e3-407a-a72b-298b4cd01349 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:21.772311 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1c1e5-e8e3-407a-a72b-298b4cd01349 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:21.773741 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1c1e5-e8e3-407a-a72b-298b4cd01349 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:21.783510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1433/2868] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:22.308871 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8394e5-25f9-459f-b8ec-311b3385bc3e service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:22.318911 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8394e5-25f9-459f-b8ec-311b3385bc3e service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:22.322793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611641bd-bc37-4ceb-a4b6-ced149aa0d26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bbddf0ea88049aa9bb6a257e1f01ffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:22.323430 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611641bd-bc37-4ceb-a4b6-ced149aa0d26 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:22.323715 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611641bd-bc37-4ceb-a4b6-ced149aa0d26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:22.323991 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611641bd-bc37-4ceb-a4b6-ced149aa0d26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:22.333691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611641bd-bc37-4ceb-a4b6-ced149aa0d26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:22.335108 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611641bd-bc37-4ceb-a4b6-ced149aa0d26 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:22.344819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1436/2869] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:26.197510 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1c1e5-e8e3-407a-a72b-298b4cd01349 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:26.226560 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:26.267592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1c1e5-e8e3-407a-a72b-298b4cd01349 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:26.273091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:26.274368 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:26.274824 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:26.275370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] PATH_INFO: `/v3/users/992423d4fe5240aca76d51a15a7d6f01` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:26.276974 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=992423d4fe5240aca76d51a15a7d6f01)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:26.278853 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:26.325996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992423d4fe5240aca76d51a15a7d6f01'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:26.327364 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992423d4fe5240aca76d51a15a7d6f01'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:26.328109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992423d4fe5240aca76d51a15a7d6f01'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:26.334964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992423d4fe5240aca76d51a15a7d6f01'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:26.346769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992423d4fe5240aca76d51a15a7d6f01'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:26.359571 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1434/2870] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:14:26 2022] DELETE /identity/v3/users/992423d4fe5240aca76d51a15a7d6f01 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:26.364030 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4107c10-f1b9-4092-9763-1b284cf56181 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:26.364532 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4107c10-f1b9-4092-9763-1b284cf56181 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:26.364644 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4107c10-f1b9-4092-9763-1b284cf56181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:26.382363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f4107c10-f1b9-4092-9763-1b284cf56181 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:26.383545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f4107c10-f1b9-4092-9763-1b284cf56181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:26.447980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1437/2871] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:26.458130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:26.459532 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:26.515712 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d729a3-31ae-4300-800d-ecb470c58b48 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:26.519592 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32c083-e854-4870-9587-f12e48f8b403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:26.520259 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c083-e854-4870-9587-f12e48f8b403 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:26.520565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c083-e854-4870-9587-f12e48f8b403 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:26.520845 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c083-e854-4870-9587-f12e48f8b403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:26.530459 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32c083-e854-4870-9587-f12e48f8b403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:26.531892 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32c083-e854-4870-9587-f12e48f8b403 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:26.541338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1435/2872] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:26.817251 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4107c10-f1b9-4092-9763-1b284cf56181 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:26.818700 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f4107c10-f1b9-4092-9763-1b284cf56181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:26.831538 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4107c10-f1b9-4092-9763-1b284cf56181 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:26.836794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:26.837673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:26.837950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:26.838227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] PATH_INFO: `/v3/projects/04c8dea3ca3a4c679c98997b6beff7cf` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:26.839399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c8dea3ca3a4c679c98997b6beff7cf)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:26.841333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:26.888820 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c8dea3ca3a4c679c98997b6beff7cf', 'request_id': 'req-9f8d3d88-cdfc-4c5b-be84-1babca116750'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:26.896096 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c8dea3ca3a4c679c98997b6beff7cf', 'request_id': 'req-9f8d3d88-cdfc-4c5b-be84-1babca116750'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:26.903031 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c8dea3ca3a4c679c98997b6beff7cf', 'request_id': 'req-9f8d3d88-cdfc-4c5b-be84-1babca116750'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:26.918917 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] The token cache is being invalidate because project 04c8dea3ca3a4c679c98997b6beff7cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:14:26.919347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:26.920577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:26.922828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1438/2873] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:14:26 2022] DELETE /identity/v3/projects/04c8dea3ca3a4c679c98997b6beff7cf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:26.926702 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32c083-e854-4870-9587-f12e48f8b403 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:26.938634 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32c083-e854-4870-9587-f12e48f8b403 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:26.981423 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:26.997191 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:26.998091 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:26.998933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:26.999676 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] PATH_INFO: `/v3/users/163bca3c3f4f4c28ae79ae1090ec7ef8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:27.000980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] RBAC: Authorizing `identity:delete_user(user_id=163bca3c3f4f4c28ae79ae1090ec7ef8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:27.003114 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:27.050185 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '163bca3c3f4f4c28ae79ae1090ec7ef8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.050832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '163bca3c3f4f4c28ae79ae1090ec7ef8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.051428 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '163bca3c3f4f4c28ae79ae1090ec7ef8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.058149 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '163bca3c3f4f4c28ae79ae1090ec7ef8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.070574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '163bca3c3f4f4c28ae79ae1090ec7ef8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.084268 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1436/2874] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:14:26 2022] DELETE /identity/v3/users/163bca3c3f4f4c28ae79ae1090ec7ef8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 18:14:27.366424 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:27.367935 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:27.383333 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d3d88-cdfc-4c5b-be84-1babca116750 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:27.388517 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:27.443489 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:27.444389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:27.444671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:27.444950 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] PATH_INFO: `/v3/projects/0b5c2ee8ca0841e2a26efff9fab20162` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:27.446119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b5c2ee8ca0841e2a26efff9fab20162)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:27.448017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:27.494295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b5c2ee8ca0841e2a26efff9fab20162', 'request_id': 'req-07d91e01-1a7b-4031-b49f-b1d26822ff80'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.500902 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b5c2ee8ca0841e2a26efff9fab20162', 'request_id': 'req-07d91e01-1a7b-4031-b49f-b1d26822ff80'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.508250 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b5c2ee8ca0841e2a26efff9fab20162', 'request_id': 'req-07d91e01-1a7b-4031-b49f-b1d26822ff80'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.519493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] The token cache is being invalidate because project 0b5c2ee8ca0841e2a26efff9fab20162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:14:27.519779 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.520474 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.522009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1439/2875] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:14:27 2022] DELETE /identity/v3/projects/0b5c2ee8ca0841e2a26efff9fab20162 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:27.525931 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:27.527338 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:27.541942 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8b8bf-5bf7-440a-8cc0-46fbfae4e58b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:27.602118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:27.602996 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:27.603278 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:27.603550 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] PATH_INFO: `/v3/users/6e4fbdceec91417a950303237516c70a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:27.604774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4fbdceec91417a950303237516c70a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:27.606687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:27.657342 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4fbdceec91417a950303237516c70a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.657999 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4fbdceec91417a950303237516c70a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.658601 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4fbdceec91417a950303237516c70a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.665384 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4fbdceec91417a950303237516c70a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.673270 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:27.679555 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ceffe150-c736-4679-a529-eaa86b793fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4fbdceec91417a950303237516c70a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:27.692890 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1437/2876] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:14:27 2022] DELETE /identity/v3/users/6e4fbdceec91417a950303237516c70a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:27.970035 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:27.971481 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:27.993420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d91e01-1a7b-4031-b49f-b1d26822ff80 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:28.054329 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:28.055144 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.055453 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.055756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] PATH_INFO: `/v3/projects/8142c7421b1f436eb720685bb0733815` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.057127 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8142c7421b1f436eb720685bb0733815)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:28.059247 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:28.107536 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:28.115658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8142c7421b1f436eb720685bb0733815', 'request_id': 'req-817d77f7-e4ff-4d79-af61-0234f588e200'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:28.122216 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8142c7421b1f436eb720685bb0733815', 'request_id': 'req-817d77f7-e4ff-4d79-af61-0234f588e200'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:28.129520 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8142c7421b1f436eb720685bb0733815', 'request_id': 'req-817d77f7-e4ff-4d79-af61-0234f588e200'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:28.140408 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] The token cache is being invalidate because project 8142c7421b1f436eb720685bb0733815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:14:28.140653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:28.141242 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-817d77f7-e4ff-4d79-af61-0234f588e200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:14:28.142947 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1440/2877] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:14:27 2022] DELETE /identity/v3/projects/8142c7421b1f436eb720685bb0733815 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:28.147804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe1415-5b3a-49f0-9e70-386382edac24 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.148094 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe1415-5b3a-49f0-9e70-386382edac24 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.148424 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe1415-5b3a-49f0-9e70-386382edac24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.166766 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cfbe1415-5b3a-49f0-9e70-386382edac24 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:28.167964 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cfbe1415-5b3a-49f0-9e70-386382edac24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:28.227790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1438/2878] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:28.236888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba977345-f2ef-4985-a8bd-0c2cd04239ce None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.237119 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba977345-f2ef-4985-a8bd-0c2cd04239ce None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.237399 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba977345-f2ef-4985-a8bd-0c2cd04239ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.256322 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ba977345-f2ef-4985-a8bd-0c2cd04239ce None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:28.257540 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ba977345-f2ef-4985-a8bd-0c2cd04239ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:28.265561 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1441/2879] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:28.270343 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe1415-5b3a-49f0-9e70-386382edac24 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:28.280427 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe1415-5b3a-49f0-9e70-386382edac24 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:28.287531 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00191e4-eb49-40d5-bd74-475b98dbac4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:28.288184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00191e4-eb49-40d5-bd74-475b98dbac4c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.288499 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00191e4-eb49-40d5-bd74-475b98dbac4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.288774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00191e4-eb49-40d5-bd74-475b98dbac4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.289508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00191e4-eb49-40d5-bd74-475b98dbac4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:14:28.289818 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00191e4-eb49-40d5-bd74-475b98dbac4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:28.291397 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00191e4-eb49-40d5-bd74-475b98dbac4c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:28.305885 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1439/2880] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:14:28.309953 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba977345-f2ef-4985-a8bd-0c2cd04239ce None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:28.319743 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba977345-f2ef-4985-a8bd-0c2cd04239ce None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:28.324785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb9979-6a41-4dcc-abb8-68aa4f7a55b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:28.325468 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb9979-6a41-4dcc-abb8-68aa4f7a55b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.325795 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb9979-6a41-4dcc-abb8-68aa4f7a55b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.326104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb9979-6a41-4dcc-abb8-68aa4f7a55b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.327558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb9979-6a41-4dcc-abb8-68aa4f7a55b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:28.329496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb9979-6a41-4dcc-abb8-68aa4f7a55b6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:28.355283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1442/2881] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:14:28 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:28.360051 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00191e4-eb49-40d5-bd74-475b98dbac4c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:28.372130 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00191e4-eb49-40d5-bd74-475b98dbac4c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:28.435985 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e4b9d-e5f7-4668-a04e-f9f2589a5dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:28.436861 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e4b9d-e5f7-4668-a04e-f9f2589a5dea admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.437148 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e4b9d-e5f7-4668-a04e-f9f2589a5dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.437426 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e4b9d-e5f7-4668-a04e-f9f2589a5dea admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.438151 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e4b9d-e5f7-4668-a04e-f9f2589a5dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:28.440045 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e4b9d-e5f7-4668-a04e-f9f2589a5dea admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:28.459075 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1440/2882] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:14:28 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:28.463602 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb9979-6a41-4dcc-abb8-68aa4f7a55b6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:28.474537 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb9979-6a41-4dcc-abb8-68aa4f7a55b6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:28.479139 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058eec-4a31-41cc-87a0-d21c560a1c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:28.479793 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058eec-4a31-41cc-87a0-d21c560a1c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.480078 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058eec-4a31-41cc-87a0-d21c560a1c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.480397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058eec-4a31-41cc-87a0-d21c560a1c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.481135 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69058eec-4a31-41cc-87a0-d21c560a1c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:28.482813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69058eec-4a31-41cc-87a0-d21c560a1c64 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:28.493797 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1443/2883] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 18:14:28.498731 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e4b9d-e5f7-4668-a04e-f9f2589a5dea admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:28.510201 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e4b9d-e5f7-4668-a04e-f9f2589a5dea admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:28.514883 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d85d86-7979-483c-bf7e-db27ec2a1181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:28.515545 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d85d86-7979-483c-bf7e-db27ec2a1181 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.515826 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d85d86-7979-483c-bf7e-db27ec2a1181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.516104 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d85d86-7979-483c-bf7e-db27ec2a1181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.516873 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d85d86-7979-483c-bf7e-db27ec2a1181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:28.518454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d85d86-7979-483c-bf7e-db27ec2a1181 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:28.530933 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1441/2884] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:28.534979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058eec-4a31-41cc-87a0-d21c560a1c64 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:28.546514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058eec-4a31-41cc-87a0-d21c560a1c64 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:28.551067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f97db-1634-4a10-a948-cdaf00f81654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:28.551937 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f97db-1634-4a10-a948-cdaf00f81654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.552244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f97db-1634-4a10-a948-cdaf00f81654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.552550 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f97db-1634-4a10-a948-cdaf00f81654 admin admin] PATH_INFO: `/v3/projects/62222b635de744939d6e21288baa4f67/users/141b69f40b5d41debdc4585787c3d47c/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.567787 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f97db-1634-4a10-a948-cdaf00f81654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62222b635de744939d6e21288baa4f67, user_id=141b69f40b5d41debdc4585787c3d47c, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:28.570386 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f97db-1634-4a10-a948-cdaf00f81654 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:28.583629 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1444/2885] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:28 2022] PUT /identity/v3/projects/62222b635de744939d6e21288baa4f67/users/141b69f40b5d41debdc4585787c3d47c/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:28.588744 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab47a-83ee-423c-afc5-901d5e29c666 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.589037 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab47a-83ee-423c-afc5-901d5e29c666 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.589316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab47a-83ee-423c-afc5-901d5e29c666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.607115 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-98fab47a-83ee-423c-afc5-901d5e29c666 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:28.608878 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-98fab47a-83ee-423c-afc5-901d5e29c666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:28.653642 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:28.674382 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1442/2886] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:28.684024 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f97db-1634-4a10-a948-cdaf00f81654 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:28.685786 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4b3f97db-1634-4a10-a948-cdaf00f81654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:28.743665 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f97db-1634-4a10-a948-cdaf00f81654 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:28.747423 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd2465-0308-486a-9614-e1690619a40d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:28.748066 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd2465-0308-486a-9614-e1690619a40d service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:28.748979 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd2465-0308-486a-9614-e1690619a40d service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:28.749416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd2465-0308-486a-9614-e1690619a40d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:28.758542 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd2465-0308-486a-9614-e1690619a40d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:28.759972 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd2465-0308-486a-9614-e1690619a40d service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:28.769339 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1445/2887] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:32.679083 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2dc67-9803-462f-8f9f-f9a8c8e91b93 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:32.679559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2dc67-9803-462f-8f9f-f9a8c8e91b93 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:32.679648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2dc67-9803-462f-8f9f-f9a8c8e91b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:32.709311 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-52b2dc67-9803-462f-8f9f-f9a8c8e91b93 None None] MFA Rules not processed for user `141b69f40b5d41debdc4585787c3d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:32.710528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-52b2dc67-9803-462f-8f9f-f9a8c8e91b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:32.775949 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1443/2888] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:32.780488 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd2465-0308-486a-9614-e1690619a40d service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:32.790505 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd2465-0308-486a-9614-e1690619a40d service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:32.796344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165b2e6-21ec-45b0-b3b0-b068f025ab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:32.797017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165b2e6-21ec-45b0-b3b0-b068f025ab9b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:32.797300 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165b2e6-21ec-45b0-b3b0-b068f025ab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:32.797577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165b2e6-21ec-45b0-b3b0-b068f025ab9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:32.798293 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165b2e6-21ec-45b0-b3b0-b068f025ab9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:32.800207 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165b2e6-21ec-45b0-b3b0-b068f025ab9b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:32.831555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1446/2889] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:14:32 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:32.833117 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2dc67-9803-462f-8f9f-f9a8c8e91b93 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:32.846078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2dc67-9803-462f-8f9f-f9a8c8e91b93 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:32.906973 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d70390-11d2-415c-9e3b-804ee31f344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:32.907655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d70390-11d2-415c-9e3b-804ee31f344b admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:32.907940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d70390-11d2-415c-9e3b-804ee31f344b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:32.908219 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d70390-11d2-415c-9e3b-804ee31f344b admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:32.908980 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d70390-11d2-415c-9e3b-804ee31f344b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:32.910918 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d70390-11d2-415c-9e3b-804ee31f344b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:32.931060 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1444/2890] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:14:32 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:14:32.934982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165b2e6-21ec-45b0-b3b0-b068f025ab9b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:32.946038 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165b2e6-21ec-45b0-b3b0-b068f025ab9b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:32.950594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb37d6-fd62-4e22-be2e-18c5d35b34b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:32.951904 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb37d6-fd62-4e22-be2e-18c5d35b34b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:32.952184 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb37d6-fd62-4e22-be2e-18c5d35b34b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:32.952496 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb37d6-fd62-4e22-be2e-18c5d35b34b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:32.953203 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb37d6-fd62-4e22-be2e-18c5d35b34b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:32.955533 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb37d6-fd62-4e22-be2e-18c5d35b34b2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:32.967730 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1447/2891] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:32.971838 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d70390-11d2-415c-9e3b-804ee31f344b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:32.982483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d70390-11d2-415c-9e3b-804ee31f344b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:32.987606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b801fb-0247-440e-9827-f23b455dcad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:32.988542 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b801fb-0247-440e-9827-f23b455dcad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:32.988841 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b801fb-0247-440e-9827-f23b455dcad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:32.989107 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b801fb-0247-440e-9827-f23b455dcad5 admin admin] PATH_INFO: `/v3/projects/acafd58a1eec47c9afc44b5ead8e5084/users/f2354d08573a414486adc93d80c57ef4/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:33.007128 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b801fb-0247-440e-9827-f23b455dcad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acafd58a1eec47c9afc44b5ead8e5084, user_id=f2354d08573a414486adc93d80c57ef4, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:33.009874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b801fb-0247-440e-9827-f23b455dcad5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:33.033325 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1445/2892] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:32 2022] PUT /identity/v3/projects/acafd58a1eec47c9afc44b5ead8e5084/users/f2354d08573a414486adc93d80c57ef4/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:33.034577 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb37d6-fd62-4e22-be2e-18c5d35b34b2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:33.049227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb37d6-fd62-4e22-be2e-18c5d35b34b2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:33.087740 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:33.115501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685d1a4-e290-40bd-88a5-70f7fb2f76cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:33.116104 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685d1a4-e290-40bd-88a5-70f7fb2f76cf admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:33.116429 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685d1a4-e290-40bd-88a5-70f7fb2f76cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:33.116709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685d1a4-e290-40bd-88a5-70f7fb2f76cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:33.117435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685d1a4-e290-40bd-88a5-70f7fb2f76cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:33.119062 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685d1a4-e290-40bd-88a5-70f7fb2f76cf admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:33.130709 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1448/2893] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:14:33.134797 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b801fb-0247-440e-9827-f23b455dcad5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:33.146328 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b801fb-0247-440e-9827-f23b455dcad5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:33.155562 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73028552-5984-49ee-8be3-b9dfed20a5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:33.156605 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73028552-5984-49ee-8be3-b9dfed20a5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:33.156954 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73028552-5984-49ee-8be3-b9dfed20a5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:33.157298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73028552-5984-49ee-8be3-b9dfed20a5fa admin admin] PATH_INFO: `/v3/projects/acafd58a1eec47c9afc44b5ead8e5084/users/f2354d08573a414486adc93d80c57ef4/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:33.159517 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73028552-5984-49ee-8be3-b9dfed20a5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=acafd58a1eec47c9afc44b5ead8e5084, user_id=f2354d08573a414486adc93d80c57ef4, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:33.162198 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73028552-5984-49ee-8be3-b9dfed20a5fa admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:33.173752 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1446/2894] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:14:33 2022] PUT /identity/v3/projects/acafd58a1eec47c9afc44b5ead8e5084/users/f2354d08573a414486adc93d80c57ef4/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:14:37.519884 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1ed1d-ff69-4eed-b763-42ad97508795 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:37.520382 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1ed1d-ff69-4eed-b763-42ad97508795 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:37.520576 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1ed1d-ff69-4eed-b763-42ad97508795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:37.550033 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-34d1ed1d-ff69-4eed-b763-42ad97508795 None None] MFA Rules not processed for user `f2354d08573a414486adc93d80c57ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:37.551498 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-34d1ed1d-ff69-4eed-b763-42ad97508795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:37.640342 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1449/2895] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:37.651776 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73028552-5984-49ee-8be3-b9dfed20a5fa admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:37.653832 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-73028552-5984-49ee-8be3-b9dfed20a5fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:37.693199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:37.723658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73028552-5984-49ee-8be3-b9dfed20a5fa admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:37.727171 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507389e1-386d-4590-aaa9-a28ebfe93115 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e07c59babad479cadbf74f15ee5eea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:37.727854 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507389e1-386d-4590-aaa9-a28ebfe93115 service glance] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:37.728256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507389e1-386d-4590-aaa9-a28ebfe93115 service glance] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:37.728572 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507389e1-386d-4590-aaa9-a28ebfe93115 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:37.789193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507389e1-386d-4590-aaa9-a28ebfe93115 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:37.790723 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507389e1-386d-4590-aaa9-a28ebfe93115 service glance] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:37.801646 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1447/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:37.917217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39973c-161b-4b01-b541-65e83438a669 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:37.917217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39973c-161b-4b01-b541-65e83438a669 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:37.917217 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39973c-161b-4b01-b541-65e83438a669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:37.933992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ba39973c-161b-4b01-b541-65e83438a669 None None] MFA Rules not processed for user `75f11eaa4b504d0aa4018fb4ffbbd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:37.935227 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ba39973c-161b-4b01-b541-65e83438a669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:38.003914 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1450/2897] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 18:14:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:38.053252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507389e1-386d-4590-aaa9-a28ebfe93115 service glance] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:38.054683 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-507389e1-386d-4590-aaa9-a28ebfe93115 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:38.127678 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507389e1-386d-4590-aaa9-a28ebfe93115 service glance] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:38.133167 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b71445-6922-4835-8af6-93063b64c658 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a26412cf9f64b1bbbd8e595b55bcebc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:38.133961 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b71445-6922-4835-8af6-93063b64c658 service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:38.134249 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b71445-6922-4835-8af6-93063b64c658 service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:38.134526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b71445-6922-4835-8af6-93063b64c658 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:38.143590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b71445-6922-4835-8af6-93063b64c658 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:38.145244 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b71445-6922-4835-8af6-93063b64c658 service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:38.153804 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1448/2898] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jul 21 18:14:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 18:14:47.917558 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb5fa8-0e49-476a-a2b4-d71adf0dfae1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:47.918044 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb5fa8-0e49-476a-a2b4-d71adf0dfae1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:47.918133 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb5fa8-0e49-476a-a2b4-d71adf0dfae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:47.936860 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e1cb5fa8-0e49-476a-a2b4-d71adf0dfae1 None None] MFA Rules not processed for user `141b69f40b5d41debdc4585787c3d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:47.938032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e1cb5fa8-0e49-476a-a2b4-d71adf0dfae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:47.945035 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1451/2899] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:14:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:47.985815 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b71445-6922-4835-8af6-93063b64c658 service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:47.995031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b71445-6922-4835-8af6-93063b64c658 service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:47.998660 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a9798-893d-43e3-ad7d-6966d89577ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e07c59babad479cadbf74f15ee5eea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:47.999295 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a9798-893d-43e3-ad7d-6966d89577ff service glance] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:47.999580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a9798-893d-43e3-ad7d-6966d89577ff service glance] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:47.999857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a9798-893d-43e3-ad7d-6966d89577ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:48.001653 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3c3a9798-893d-43e3-ad7d-6966d89577ff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:48.071874 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a9798-893d-43e3-ad7d-6966d89577ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:48.073243 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a9798-893d-43e3-ad7d-6966d89577ff service glance] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:48.079044 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:48.094745 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1449/2900] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:48.625842 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb5fa8-0e49-476a-a2b4-d71adf0dfae1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:48.679684 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:48.688122 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb5fa8-0e49-476a-a2b4-d71adf0dfae1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:48.691836 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837165a-2c4c-4f02-9450-a36663262328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:48.692523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837165a-2c4c-4f02-9450-a36663262328 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:48.692813 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837165a-2c4c-4f02-9450-a36663262328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:48.693095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837165a-2c4c-4f02-9450-a36663262328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:48.694897 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c837165a-2c4c-4f02-9450-a36663262328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:48.709534 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837165a-2c4c-4f02-9450-a36663262328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:48.710975 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837165a-2c4c-4f02-9450-a36663262328 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:48.722347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1452/2901] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:14:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:14:49.841844 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93355b3-4fc2-44fe-a9d9-f74df131c13f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:49.843015 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93355b3-4fc2-44fe-a9d9-f74df131c13f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:49.843520 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93355b3-4fc2-44fe-a9d9-f74df131c13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:49.863913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e93355b3-4fc2-44fe-a9d9-f74df131c13f None None] MFA Rules not processed for user `75f11eaa4b504d0aa4018fb4ffbbd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:14:49.865220 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e93355b3-4fc2-44fe-a9d9-f74df131c13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:14:49.873210 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1450/2902] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Jul 21 18:14:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:14:49.885370 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837165a-2c4c-4f02-9450-a36663262328 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:14:49.897539 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837165a-2c4c-4f02-9450-a36663262328 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:14:49.904269 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8232f-e4ca-4028-a1aa-aac323caaff6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a26412cf9f64b1bbbd8e595b55bcebc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:14:49.905004 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8232f-e4ca-4028-a1aa-aac323caaff6 service swift] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:14:49.905345 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8232f-e4ca-4028-a1aa-aac323caaff6 service swift] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:14:49.905683 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8232f-e4ca-4028-a1aa-aac323caaff6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:14:49.916340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8232f-e4ca-4028-a1aa-aac323caaff6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:14:49.916340 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8232f-e4ca-4028-a1aa-aac323caaff6 service swift] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:14:49.925878 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1453/2903] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jul 21 18:14:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 18:14:59.858371 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:14:59.922334 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=10 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:15:17.705448 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93355b3-4fc2-44fe-a9d9-f74df131c13f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:15:17.714650 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93355b3-4fc2-44fe-a9d9-f74df131c13f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:15:17.719546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45222231-953d-4b72-9e07-599659990ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:15:17.720182 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45222231-953d-4b72-9e07-599659990ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:15:17.720494 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45222231-953d-4b72-9e07-599659990ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:15:17.720771 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45222231-953d-4b72-9e07-599659990ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:15:17.729482 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45222231-953d-4b72-9e07-599659990ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:15:17.730890 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45222231-953d-4b72-9e07-599659990ac7 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:15:17.740763 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1451/2904] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:15:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:15:19.485270 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8232f-e4ca-4028-a1aa-aac323caaff6 service swift] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:15:19.486805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9da8232f-e4ca-4028-a1aa-aac323caaff6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:15:19.505728 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8232f-e4ca-4028-a1aa-aac323caaff6 service swift] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:15:19.511344 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9873212-65b9-4c67-a5cb-06fe4d368a38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:15:19.511888 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9873212-65b9-4c67-a5cb-06fe4d368a38 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:15:19.512168 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9873212-65b9-4c67-a5cb-06fe4d368a38 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:15:19.512785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9873212-65b9-4c67-a5cb-06fe4d368a38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:15:19.514514 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b9873212-65b9-4c67-a5cb-06fe4d368a38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:15:19.529545 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9873212-65b9-4c67-a5cb-06fe4d368a38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:15:19.530129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9873212-65b9-4c67-a5cb-06fe4d368a38 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:15:19.555984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1454/2905] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:15:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:15:27.736718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:15:29.538666 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:15:31.678780 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45222231-953d-4b72-9e07-599659990ac7 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:15:31.697200 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45222231-953d-4b72-9e07-599659990ac7 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:15:31.701395 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db2c87-dacd-4a03-a4b7-1c50002e989d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:15:31.702030 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db2c87-dacd-4a03-a4b7-1c50002e989d service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:15:31.702315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db2c87-dacd-4a03-a4b7-1c50002e989d service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:15:31.702595 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db2c87-dacd-4a03-a4b7-1c50002e989d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:15:31.704400 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-19db2c87-dacd-4a03-a4b7-1c50002e989d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:15:31.768664 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db2c87-dacd-4a03-a4b7-1c50002e989d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:15:31.770074 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db2c87-dacd-4a03-a4b7-1c50002e989d service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:15:31.780616 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1452/2906] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:15:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:15:34.808639 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9873212-65b9-4c67-a5cb-06fe4d368a38 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:15:34.810346 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b9873212-65b9-4c67-a5cb-06fe4d368a38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:15:34.887480 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9873212-65b9-4c67-a5cb-06fe4d368a38 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:15:34.893493 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc8924-2fe9-4041-a232-7e0ab8c56991 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:15:34.894154 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc8924-2fe9-4041-a232-7e0ab8c56991 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:15:34.894438 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc8924-2fe9-4041-a232-7e0ab8c56991 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:15:34.894714 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc8924-2fe9-4041-a232-7e0ab8c56991 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:15:34.896516 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-26bc8924-2fe9-4041-a232-7e0ab8c56991 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:15:34.910040 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc8924-2fe9-4041-a232-7e0ab8c56991 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:15:34.911566 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc8924-2fe9-4041-a232-7e0ab8c56991 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:15:34.923555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1455/2907] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:15:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:15:37.992599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db2c87-dacd-4a03-a4b7-1c50002e989d service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:15:37.994290 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-19db2c87-dacd-4a03-a4b7-1c50002e989d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:15:38.009739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db2c87-dacd-4a03-a4b7-1c50002e989d service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:15:38.015372 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:15:38.016234 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:15:38.016565 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:15:38.016839 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] PATH_INFO: `/v3/users/293d4ca894e24e4fb6c4e7aa14256d78` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:15:38.018004 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=293d4ca894e24e4fb6c4e7aa14256d78)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:15:38.019895 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:15:38.066667 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '293d4ca894e24e4fb6c4e7aa14256d78'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.067477 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '293d4ca894e24e4fb6c4e7aa14256d78'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.068193 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '293d4ca894e24e4fb6c4e7aa14256d78'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.076691 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '293d4ca894e24e4fb6c4e7aa14256d78'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.088658 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '293d4ca894e24e4fb6c4e7aa14256d78'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.101418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1453/2908] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:15:37 2022] DELETE /identity/v3/users/293d4ca894e24e4fb6c4e7aa14256d78 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:15:38.273814 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc8924-2fe9-4041-a232-7e0ab8c56991 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:15:38.286603 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc8924-2fe9-4041-a232-7e0ab8c56991 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:15:38.346664 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:15:38.347521 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:15:38.347800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:15:38.348073 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] PATH_INFO: `/v3/projects/076a5b3e4cd141b9bd9ef199e0fa0ba1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:15:38.349264 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=076a5b3e4cd141b9bd9ef199e0fa0ba1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:15:38.351463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:15:38.398470 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076a5b3e4cd141b9bd9ef199e0fa0ba1', 'request_id': 'req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.404753 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076a5b3e4cd141b9bd9ef199e0fa0ba1', 'request_id': 'req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.412772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076a5b3e4cd141b9bd9ef199e0fa0ba1', 'request_id': 'req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.424079 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] The token cache is being invalidate because project 076a5b3e4cd141b9bd9ef199e0fa0ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:15:38.424397 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.425020 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.426560 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1456/2909] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:15:38 2022] DELETE /identity/v3/projects/076a5b3e4cd141b9bd9ef199e0fa0ba1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:15:38.430466 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:15:38.431852 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:15:38.445320 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b58f1-53ad-4420-949a-491a46bf7563 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:15:38.495534 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:15:38.503001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:15:38.503867 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:15:38.504543 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:15:38.505199 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] PATH_INFO: `/v3/users/fd67132c544c46c9a7d82f55128bdcb3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:15:38.506640 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd67132c544c46c9a7d82f55128bdcb3)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:15:38.509026 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:15:38.554147 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd67132c544c46c9a7d82f55128bdcb3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.554801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd67132c544c46c9a7d82f55128bdcb3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.555407 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd67132c544c46c9a7d82f55128bdcb3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.563011 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd67132c544c46c9a7d82f55128bdcb3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.574587 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd67132c544c46c9a7d82f55128bdcb3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.594661 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1454/2910] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:15:38 2022] DELETE /identity/v3/users/fd67132c544c46c9a7d82f55128bdcb3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:15:38.764831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:15:38.766427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:15:38.780876 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b033f7-59fd-4b5e-a3a7-191afcf87a4b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:15:38.801012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:15:38.842158 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:15:38.843017 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:15:38.843295 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:15:38.843564 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] PATH_INFO: `/v3/projects/c77d072230894424a78dc0a43457c8da` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:15:38.844815 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c77d072230894424a78dc0a43457c8da)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:15:38.846731 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:15:38.895358 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c77d072230894424a78dc0a43457c8da', 'request_id': 'req-6c6a2e9d-096f-4586-a09f-f45e21801266'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.901750 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c77d072230894424a78dc0a43457c8da', 'request_id': 'req-6c6a2e9d-096f-4586-a09f-f45e21801266'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.909819 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c77d072230894424a78dc0a43457c8da', 'request_id': 'req-6c6a2e9d-096f-4586-a09f-f45e21801266'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.921693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] The token cache is being invalidate because project c77d072230894424a78dc0a43457c8da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:15:38.921978 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.922623 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6c6a2e9d-096f-4586-a09f-f45e21801266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:15:38.924172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1457/2911] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:15:38 2022] DELETE /identity/v3/projects/c77d072230894424a78dc0a43457c8da => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:15:44.220597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:15:44.221965 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:15:44.293064 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c461b4-6a72-474f-9ef7-749b67f270b8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:15:44.349129 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236f425-ce85-4d45-a2af-54a685a0724b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:15:44.349792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236f425-ce85-4d45-a2af-54a685a0724b service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:15:44.350072 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236f425-ce85-4d45-a2af-54a685a0724b service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:15:44.350348 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236f425-ce85-4d45-a2af-54a685a0724b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:15:44.352240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b236f425-ce85-4d45-a2af-54a685a0724b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:15:44.362172 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236f425-ce85-4d45-a2af-54a685a0724b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:15:44.363575 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236f425-ce85-4d45-a2af-54a685a0724b service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:15:44.371756 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1455/2912] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:15:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:15:48.917893 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:15:54.367857 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:16:40.213968 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b682671-0f5d-4bb6-b581-b1c54020a2a5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:40.214412 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b682671-0f5d-4bb6-b581-b1c54020a2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:40.214694 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b682671-0f5d-4bb6-b581-b1c54020a2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:40.233359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4b682671-0f5d-4bb6-b581-b1c54020a2a5 None None] MFA Rules not processed for user `75f11eaa4b504d0aa4018fb4ffbbd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:16:40.234618 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4b682671-0f5d-4bb6-b581-b1c54020a2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:40.288784 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1458/2913] 10.222.0.57 () {58 vars in 967 bytes} [Thu Jul 21 18:16:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:16:40.329828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236f425-ce85-4d45-a2af-54a685a0724b service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:40.331205 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b236f425-ce85-4d45-a2af-54a685a0724b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:40.398759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236f425-ce85-4d45-a2af-54a685a0724b service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:40.403604 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d65c60-debd-4999-96cb-345e0ae05c97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a26412cf9f64b1bbbd8e595b55bcebc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:40.404236 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d65c60-debd-4999-96cb-345e0ae05c97 service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:40.404628 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d65c60-debd-4999-96cb-345e0ae05c97 service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:40.404904 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d65c60-debd-4999-96cb-345e0ae05c97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:40.413099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d65c60-debd-4999-96cb-345e0ae05c97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:40.414598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d65c60-debd-4999-96cb-345e0ae05c97 service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:40.422730 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1456/2914] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Jul 21 18:16:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 18:16:46.900051 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b682671-0f5d-4bb6-b581-b1c54020a2a5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:46.901526 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4b682671-0f5d-4bb6-b581-b1c54020a2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:46.973420 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b682671-0f5d-4bb6-b581-b1c54020a2a5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:46.977982 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:46.978846 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:46.979129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:46.979407 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] PATH_INFO: `/v3/users/141b69f40b5d41debdc4585787c3d47c` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:46.980636 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=141b69f40b5d41debdc4585787c3d47c)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:46.982573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:47.028387 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '141b69f40b5d41debdc4585787c3d47c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.035475 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '141b69f40b5d41debdc4585787c3d47c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.036258 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '141b69f40b5d41debdc4585787c3d47c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.048823 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '141b69f40b5d41debdc4585787c3d47c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.056626 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '141b69f40b5d41debdc4585787c3d47c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.062693 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1459/2915] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:16:46 2022] DELETE /identity/v3/users/141b69f40b5d41debdc4585787c3d47c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:16:47.238672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d65c60-debd-4999-96cb-345e0ae05c97 service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:47.252076 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d65c60-debd-4999-96cb-345e0ae05c97 service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:47.311870 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:47.312759 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:47.313042 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:47.313316 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] PATH_INFO: `/v3/projects/62222b635de744939d6e21288baa4f67` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:47.314526 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62222b635de744939d6e21288baa4f67)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:47.316707 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:47.361820 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62222b635de744939d6e21288baa4f67', 'request_id': 'req-2223f405-9174-44ee-98a5-984b9dfdab62'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.368753 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62222b635de744939d6e21288baa4f67', 'request_id': 'req-2223f405-9174-44ee-98a5-984b9dfdab62'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.374574 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62222b635de744939d6e21288baa4f67', 'request_id': 'req-2223f405-9174-44ee-98a5-984b9dfdab62'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.386141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] The token cache is being invalidate because project 62222b635de744939d6e21288baa4f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:16:47.386422 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.387049 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.388590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1457/2916] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:16:47 2022] DELETE /identity/v3/projects/62222b635de744939d6e21288baa4f67 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:16:47.392555 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:47.394435 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:47.407906 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb124c9-468d-4bee-bb9f-7111e62023d4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:47.429283 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:16:47.470107 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:47.471016 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:47.471304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:47.471586 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] PATH_INFO: `/v3/users/f2354d08573a414486adc93d80c57ef4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:47.473486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2354d08573a414486adc93d80c57ef4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:47.475437 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:47.526760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2354d08573a414486adc93d80c57ef4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.533557 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2354d08573a414486adc93d80c57ef4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.534241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2354d08573a414486adc93d80c57ef4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.547960 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2354d08573a414486adc93d80c57ef4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.555984 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1684bcf-2ef6-455d-8529-a49c3866f1fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2354d08573a414486adc93d80c57ef4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.561988 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1460/2917] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:16:47 2022] DELETE /identity/v3/users/f2354d08573a414486adc93d80c57ef4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:16:47.735585 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:47.737052 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:47.750302 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223f405-9174-44ee-98a5-984b9dfdab62 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:47.767590 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:16:47.810625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:47.811483 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:47.811769 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:47.812047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] PATH_INFO: `/v3/projects/acafd58a1eec47c9afc44b5ead8e5084` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:47.813256 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acafd58a1eec47c9afc44b5ead8e5084)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:47.815158 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:47.860431 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acafd58a1eec47c9afc44b5ead8e5084', 'request_id': 'req-30e5cc75-45f7-46f6-acc4-91d754b60df9'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.868109 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acafd58a1eec47c9afc44b5ead8e5084', 'request_id': 'req-30e5cc75-45f7-46f6-acc4-91d754b60df9'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.873724 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acafd58a1eec47c9afc44b5ead8e5084', 'request_id': 'req-30e5cc75-45f7-46f6-acc4-91d754b60df9'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.884779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] The token cache is being invalidate because project acafd58a1eec47c9afc44b5ead8e5084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:16:47.885066 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.885708 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-30e5cc75-45f7-46f6-acc4-91d754b60df9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:16:47.887238 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1458/2918] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:16:47 2022] DELETE /identity/v3/projects/acafd58a1eec47c9afc44b5ead8e5084 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:16:47.892160 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11442c2c-4797-4ace-8e86-288cc220b955 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:47.892499 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11442c2c-4797-4ace-8e86-288cc220b955 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:47.892804 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11442c2c-4797-4ace-8e86-288cc220b955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:47.911772 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-11442c2c-4797-4ace-8e86-288cc220b955 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:16:47.912994 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-11442c2c-4797-4ace-8e86-288cc220b955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:47.974224 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1461/2919] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:16:47.982508 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4dca7-b19e-4988-b1ef-debef6cf1846 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:47.982799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4dca7-b19e-4988-b1ef-debef6cf1846 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:47.983080 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4dca7-b19e-4988-b1ef-debef6cf1846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:48.001781 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fbb4dca7-b19e-4988-b1ef-debef6cf1846 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:16:48.003007 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fbb4dca7-b19e-4988-b1ef-debef6cf1846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:48.011214 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1459/2920] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:16:48.015248 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11442c2c-4797-4ace-8e86-288cc220b955 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:48.025573 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11442c2c-4797-4ace-8e86-288cc220b955 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:48.030137 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e5729-f22a-4596-bb8c-eafc0aa04a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:48.030785 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e5729-f22a-4596-bb8c-eafc0aa04a35 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:48.031067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e5729-f22a-4596-bb8c-eafc0aa04a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:48.031347 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e5729-f22a-4596-bb8c-eafc0aa04a35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:48.032075 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e5729-f22a-4596-bb8c-eafc0aa04a35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 18:16:48.032421 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e5729-f22a-4596-bb8c-eafc0aa04a35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:48.034012 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e5729-f22a-4596-bb8c-eafc0aa04a35 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:48.047126 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1462/2921] 10.222.0.57 () {58 vars in 1152 bytes} [Thu Jul 21 18:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 18:16:48.051409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4dca7-b19e-4988-b1ef-debef6cf1846 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:48.060606 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4dca7-b19e-4988-b1ef-debef6cf1846 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:48.065252 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c188ee8-0927-4b8c-ac85-b12d16b922e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:48.066567 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c188ee8-0927-4b8c-ac85-b12d16b922e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:48.067463 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c188ee8-0927-4b8c-ac85-b12d16b922e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:48.067829 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c188ee8-0927-4b8c-ac85-b12d16b922e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:48.068655 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c188ee8-0927-4b8c-ac85-b12d16b922e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:48.070659 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c188ee8-0927-4b8c-ac85-b12d16b922e4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:48.096529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1460/2922] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:16:48 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:16:48.100841 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e5729-f22a-4596-bb8c-eafc0aa04a35 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:48.110653 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e5729-f22a-4596-bb8c-eafc0aa04a35 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:48.140383 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:16:48.169095 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268830e-f12a-46e9-9acc-be7583a21fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:48.169691 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268830e-f12a-46e9-9acc-be7583a21fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:48.169971 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268830e-f12a-46e9-9acc-be7583a21fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:48.170245 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268830e-f12a-46e9-9acc-be7583a21fbe admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:48.170918 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268830e-f12a-46e9-9acc-be7583a21fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:48.172851 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268830e-f12a-46e9-9acc-be7583a21fbe admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:48.192992 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1463/2923] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:16:48 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:16:48.197089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c188ee8-0927-4b8c-ac85-b12d16b922e4 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:48.207187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c188ee8-0927-4b8c-ac85-b12d16b922e4 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:48.211779 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20a394-5405-43a0-8fbb-9b94594f1b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:48.212504 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20a394-5405-43a0-8fbb-9b94594f1b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:48.212790 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20a394-5405-43a0-8fbb-9b94594f1b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:48.213069 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20a394-5405-43a0-8fbb-9b94594f1b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:48.213792 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20a394-5405-43a0-8fbb-9b94594f1b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:48.215373 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20a394-5405-43a0-8fbb-9b94594f1b2b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:48.227385 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1461/2924] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 18:16:48.231172 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268830e-f12a-46e9-9acc-be7583a21fbe admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:48.241648 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268830e-f12a-46e9-9acc-be7583a21fbe admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:48.246165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742c847-7da0-4acd-99eb-d192a71addac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:48.246938 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742c847-7da0-4acd-99eb-d192a71addac admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:48.247136 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742c847-7da0-4acd-99eb-d192a71addac admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:48.247449 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742c847-7da0-4acd-99eb-d192a71addac admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:48.248089 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742c847-7da0-4acd-99eb-d192a71addac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:48.249822 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742c847-7da0-4acd-99eb-d192a71addac admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:48.261003 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1464/2925] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:16:48.265292 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20a394-5405-43a0-8fbb-9b94594f1b2b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:48.275774 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20a394-5405-43a0-8fbb-9b94594f1b2b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:48.280454 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d914b1-d4d7-43bf-b4bf-d9167c6485f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:48.281346 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d914b1-d4d7-43bf-b4bf-d9167c6485f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:48.281625 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d914b1-d4d7-43bf-b4bf-d9167c6485f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:48.281897 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d914b1-d4d7-43bf-b4bf-d9167c6485f5 admin admin] PATH_INFO: `/v3/projects/d026c3b708b4445da50aa0be2c02a8fa/users/35918feccb574b978f06f71dc01912f2/roles/d2c2845de1d548bb9cfb46f70619b3f8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:48.297028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d914b1-d4d7-43bf-b4bf-d9167c6485f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d026c3b708b4445da50aa0be2c02a8fa, user_id=35918feccb574b978f06f71dc01912f2, role_id=d2c2845de1d548bb9cfb46f70619b3f8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:48.299630 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d914b1-d4d7-43bf-b4bf-d9167c6485f5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:48.312927 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1462/2926] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:16:48 2022] PUT /identity/v3/projects/d026c3b708b4445da50aa0be2c02a8fa/users/35918feccb574b978f06f71dc01912f2/roles/d2c2845de1d548bb9cfb46f70619b3f8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:16:48.317997 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d96bd-6947-403c-8208-971d6c199fc9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:48.318313 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d96bd-6947-403c-8208-971d6c199fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:48.318595 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d96bd-6947-403c-8208-971d6c199fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:48.336451 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a38d96bd-6947-403c-8208-971d6c199fc9 None None] MFA Rules not processed for user `57de169446f4409f8d2242abb44a08b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:16:48.337658 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a38d96bd-6947-403c-8208-971d6c199fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:48.398507 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1465/2927] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:16:48.408319 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d914b1-d4d7-43bf-b4bf-d9167c6485f5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:48.410553 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-24d914b1-d4d7-43bf-b4bf-d9167c6485f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:48.462687 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:16:48.468743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d914b1-d4d7-43bf-b4bf-d9167c6485f5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:48.474558 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbefeaa4-d527-409b-9b8f-7ef0b27c56cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:48.475207 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefeaa4-d527-409b-9b8f-7ef0b27c56cf service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:48.475493 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefeaa4-d527-409b-9b8f-7ef0b27c56cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:48.475772 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefeaa4-d527-409b-9b8f-7ef0b27c56cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:48.484718 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefeaa4-d527-409b-9b8f-7ef0b27c56cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:48.486135 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefeaa4-d527-409b-9b8f-7ef0b27c56cf service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:48.496569 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1463/2928] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:16:52.708218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bdc43-1a2b-46fa-8511-725cc78d7714 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:52.708705 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bdc43-1a2b-46fa-8511-725cc78d7714 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:52.708794 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bdc43-1a2b-46fa-8511-725cc78d7714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:52.736155 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-656bdc43-1a2b-46fa-8511-725cc78d7714 None None] MFA Rules not processed for user `35918feccb574b978f06f71dc01912f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:16:52.738025 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-656bdc43-1a2b-46fa-8511-725cc78d7714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:52.814305 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1466/2929] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:16:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:16:52.819002 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbefeaa4-d527-409b-9b8f-7ef0b27c56cf service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:52.828959 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbefeaa4-d527-409b-9b8f-7ef0b27c56cf service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:52.834462 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a6f6c-1219-477a-a9ea-cf6d427c8db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:52.835136 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a6f6c-1219-477a-a9ea-cf6d427c8db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:52.835418 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a6f6c-1219-477a-a9ea-cf6d427c8db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:52.835692 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a6f6c-1219-477a-a9ea-cf6d427c8db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:52.836433 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a6f6c-1219-477a-a9ea-cf6d427c8db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:52.838354 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a6f6c-1219-477a-a9ea-cf6d427c8db3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:52.873441 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1464/2930] 10.222.0.57 () {60 vars in 1152 bytes} [Thu Jul 21 18:16:52 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 18:16:52.877527 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bdc43-1a2b-46fa-8511-725cc78d7714 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:52.888767 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bdc43-1a2b-46fa-8511-725cc78d7714 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:52.931146 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:16:52.953771 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7145108-04a5-44d8-b541-a9ee6cb70ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:52.954452 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145108-04a5-44d8-b541-a9ee6cb70ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:52.954733 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145108-04a5-44d8-b541-a9ee6cb70ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:52.955008 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145108-04a5-44d8-b541-a9ee6cb70ee8 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:52.955711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7145108-04a5-44d8-b541-a9ee6cb70ee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:52.958290 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7145108-04a5-44d8-b541-a9ee6cb70ee8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:52.979427 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1467/2931] 10.222.0.57 () {60 vars in 1143 bytes} [Thu Jul 21 18:16:52 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 18:16:52.983415 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a6f6c-1219-477a-a9ea-cf6d427c8db3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:52.995118 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a6f6c-1219-477a-a9ea-cf6d427c8db3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:52.999995 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62e329-8afa-403a-a249-83127bfcdc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:53.000876 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62e329-8afa-403a-a249-83127bfcdc47 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:53.001163 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62e329-8afa-403a-a249-83127bfcdc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:53.001470 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62e329-8afa-403a-a249-83127bfcdc47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:53.002208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62e329-8afa-403a-a249-83127bfcdc47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:53.003825 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62e329-8afa-403a-a249-83127bfcdc47 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:53.015250 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1465/2932] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:16:53.019646 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7145108-04a5-44d8-b541-a9ee6cb70ee8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:53.029980 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7145108-04a5-44d8-b541-a9ee6cb70ee8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:53.035501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cabb88-6e3c-4476-8e6a-ded1773992ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:53.036362 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cabb88-6e3c-4476-8e6a-ded1773992ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:53.036652 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cabb88-6e3c-4476-8e6a-ded1773992ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:53.036957 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cabb88-6e3c-4476-8e6a-ded1773992ac admin admin] PATH_INFO: `/v3/projects/a4badcf42f6c4ac2ad4ab1f22617faf7/users/9b67df432e64439e8b81b105b4eb30c3/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:53.055321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cabb88-6e3c-4476-8e6a-ded1773992ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4badcf42f6c4ac2ad4ab1f22617faf7, user_id=9b67df432e64439e8b81b105b4eb30c3, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:53.058039 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cabb88-6e3c-4476-8e6a-ded1773992ac admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:53.072865 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1468/2933] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:16:53 2022] PUT /identity/v3/projects/a4badcf42f6c4ac2ad4ab1f22617faf7/users/9b67df432e64439e8b81b105b4eb30c3/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:16:53.076739 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62e329-8afa-403a-a249-83127bfcdc47 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:53.088432 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62e329-8afa-403a-a249-83127bfcdc47 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:53.163925 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26f77c-1220-459f-b75a-650b4e42dda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:53.164638 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26f77c-1220-459f-b75a-650b4e42dda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:53.164928 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26f77c-1220-459f-b75a-650b4e42dda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:53.165208 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26f77c-1220-459f-b75a-650b4e42dda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:53.165940 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26f77c-1220-459f-b75a-650b4e42dda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:53.167819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26f77c-1220-459f-b75a-650b4e42dda3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:53.177743 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1466/2934] 10.222.0.57 () {58 vars in 1121 bytes} [Thu Jul 21 18:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 18:16:53.183331 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cabb88-6e3c-4476-8e6a-ded1773992ac admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:53.198744 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cabb88-6e3c-4476-8e6a-ded1773992ac admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:53.203649 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9bf45-bf1c-4219-8ce5-e058cb4a541d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:53.204671 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9bf45-bf1c-4219-8ce5-e058cb4a541d admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:53.205018 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9bf45-bf1c-4219-8ce5-e058cb4a541d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:53.205356 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9bf45-bf1c-4219-8ce5-e058cb4a541d admin admin] PATH_INFO: `/v3/projects/a4badcf42f6c4ac2ad4ab1f22617faf7/users/9b67df432e64439e8b81b105b4eb30c3/roles/2ea917a7fe2a4acfaca8789f3c8f9857` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:53.208134 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9bf45-bf1c-4219-8ce5-e058cb4a541d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4badcf42f6c4ac2ad4ab1f22617faf7, user_id=9b67df432e64439e8b81b105b4eb30c3, role_id=2ea917a7fe2a4acfaca8789f3c8f9857)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:53.210869 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9bf45-bf1c-4219-8ce5-e058cb4a541d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:53.222477 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1469/2935] 10.222.0.57 () {60 vars in 1482 bytes} [Thu Jul 21 18:16:53 2022] PUT /identity/v3/projects/a4badcf42f6c4ac2ad4ab1f22617faf7/users/9b67df432e64439e8b81b105b4eb30c3/roles/2ea917a7fe2a4acfaca8789f3c8f9857 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:16:57.350828 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b86ed-362f-43de-aba8-9cc2231c66f8 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:57.351417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b86ed-362f-43de-aba8-9cc2231c66f8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:57.351417 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b86ed-362f-43de-aba8-9cc2231c66f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:57.378393 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-597b86ed-362f-43de-aba8-9cc2231c66f8 None None] MFA Rules not processed for user `9b67df432e64439e8b81b105b4eb30c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:16:57.379603 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-597b86ed-362f-43de-aba8-9cc2231c66f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:57.471460 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1467/2936] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:16:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:16:57.482359 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9bf45-bf1c-4219-8ce5-e058cb4a541d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:16:57.485605 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-51a9bf45-bf1c-4219-8ce5-e058cb4a541d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:57.567357 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9bf45-bf1c-4219-8ce5-e058cb4a541d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:16:57.572760 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3c809-41df-4835-8e72-e1f48ce8777f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:16:57.573404 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3c809-41df-4835-8e72-e1f48ce8777f service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:57.573682 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3c809-41df-4835-8e72-e1f48ce8777f service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:57.573956 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3c809-41df-4835-8e72-e1f48ce8777f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:57.632756 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3c809-41df-4835-8e72-e1f48ce8777f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:16:57.634559 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3c809-41df-4835-8e72-e1f48ce8777f service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:16:57.644596 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1470/2937] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:16:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:16:58.696154 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1f71e-9764-4226-adac-24d7195005b6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:16:58.696698 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1f71e-9764-4226-adac-24d7195005b6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:16:58.696793 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1f71e-9764-4226-adac-24d7195005b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:16:58.714425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d8e1f71e-9764-4226-adac-24d7195005b6 None None] MFA Rules not processed for user `35918feccb574b978f06f71dc01912f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 18:16:58.715607 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d8e1f71e-9764-4226-adac-24d7195005b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:16:58.722398 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1468/2938] 10.222.0.57 () {58 vars in 957 bytes} [Thu Jul 21 18:16:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 18:17:07.639511 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:17:08.709272 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:17:29.859905 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3c809-41df-4835-8e72-e1f48ce8777f service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:17:29.861868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-02b3c809-41df-4835-8e72-e1f48ce8777f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:17:29.918081 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3c809-41df-4835-8e72-e1f48ce8777f service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:17:29.921828 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68ac9d-94ba-4f20-8e62-4e5bfe4cf86a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:17:29.922509 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68ac9d-94ba-4f20-8e62-4e5bfe4cf86a service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:17:29.922801 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68ac9d-94ba-4f20-8e62-4e5bfe4cf86a service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:17:29.923077 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68ac9d-94ba-4f20-8e62-4e5bfe4cf86a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:17:29.933129 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68ac9d-94ba-4f20-8e62-4e5bfe4cf86a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:17:29.934548 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68ac9d-94ba-4f20-8e62-4e5bfe4cf86a service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:17:29.944974 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1471/2939] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:17:29 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:17:39.940632 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=26 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:18:40.672122 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1f71e-9764-4226-adac-24d7195005b6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:18:40.673538 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d8e1f71e-9764-4226-adac-24d7195005b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:18:40.740670 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1f71e-9764-4226-adac-24d7195005b6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:18:40.746303 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81833a2c-273a-4698-bc58-9ec8a04d04b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:18:40.746943 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81833a2c-273a-4698-bc58-9ec8a04d04b8 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:18:40.747225 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81833a2c-273a-4698-bc58-9ec8a04d04b8 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:18:40.747503 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81833a2c-273a-4698-bc58-9ec8a04d04b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:18:40.749269 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-81833a2c-273a-4698-bc58-9ec8a04d04b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:18:40.761050 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81833a2c-273a-4698-bc58-9ec8a04d04b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:18:40.762608 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81833a2c-273a-4698-bc58-9ec8a04d04b8 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:18:40.772255 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1469/2940] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:18:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:18:50.768099 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=32 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:19:39.018899 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68ac9d-94ba-4f20-8e62-4e5bfe4cf86a service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:19:39.028402 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68ac9d-94ba-4f20-8e62-4e5bfe4cf86a service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:19:39.034695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b81a7b-1920-4b7b-b1a0-6d4a6b46ab79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:19:39.035773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b81a7b-1920-4b7b-b1a0-6d4a6b46ab79 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:19:39.036067 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b81a7b-1920-4b7b-b1a0-6d4a6b46ab79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:19:39.036389 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b81a7b-1920-4b7b-b1a0-6d4a6b46ab79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:19:39.038241 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-18b81a7b-1920-4b7b-b1a0-6d4a6b46ab79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:19:39.052197 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b81a7b-1920-4b7b-b1a0-6d4a6b46ab79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:19:39.054221 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b81a7b-1920-4b7b-b1a0-6d4a6b46ab79 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:19:39.064692 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1472/2941] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:19:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:19:39.163546 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81833a2c-273a-4698-bc58-9ec8a04d04b8 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:19:39.174413 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81833a2c-273a-4698-bc58-9ec8a04d04b8 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:19:39.177570 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a66fc3-8e32-4597-8055-218b5dc72c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:19:39.178240 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a66fc3-8e32-4597-8055-218b5dc72c85 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:19:39.178529 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a66fc3-8e32-4597-8055-218b5dc72c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:19:39.178801 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a66fc3-8e32-4597-8055-218b5dc72c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:19:39.180613 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-16a66fc3-8e32-4597-8055-218b5dc72c85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:19:39.191409 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a66fc3-8e32-4597-8055-218b5dc72c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:19:39.193298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a66fc3-8e32-4597-8055-218b5dc72c85 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:19:39.201799 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1470/2942] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:19:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:19:39.320047 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b81a7b-1920-4b7b-b1a0-6d4a6b46ab79 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:19:39.329833 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b81a7b-1920-4b7b-b1a0-6d4a6b46ab79 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:19:39.334615 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266904b2-8afb-444a-a236-8eed07c221bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:19:39.336113 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266904b2-8afb-444a-a236-8eed07c221bf service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:19:39.336443 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266904b2-8afb-444a-a236-8eed07c221bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:19:39.336723 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266904b2-8afb-444a-a236-8eed07c221bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:19:39.339302 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-266904b2-8afb-444a-a236-8eed07c221bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:19:39.368396 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266904b2-8afb-444a-a236-8eed07c221bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:19:39.369831 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266904b2-8afb-444a-a236-8eed07c221bf service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:19:39.380834 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1473/2943] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:19:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:19:39.440615 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a66fc3-8e32-4597-8055-218b5dc72c85 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:19:39.452511 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a66fc3-8e32-4597-8055-218b5dc72c85 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:19:39.457966 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c15e84-09ce-4a07-a000-94db54fadc7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:19:39.458633 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15e84-09ce-4a07-a000-94db54fadc7d service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:19:39.458913 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15e84-09ce-4a07-a000-94db54fadc7d service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:19:39.459187 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15e84-09ce-4a07-a000-94db54fadc7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:19:39.460965 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b0c15e84-09ce-4a07-a000-94db54fadc7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:19:39.472669 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c15e84-09ce-4a07-a000-94db54fadc7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:19:39.474228 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c15e84-09ce-4a07-a000-94db54fadc7d service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:19:39.483916 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1471/2944] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:19:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:19:43.676907 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266904b2-8afb-444a-a236-8eed07c221bf service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:19:43.678310 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-266904b2-8afb-444a-a236-8eed07c221bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:19:43.699504 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266904b2-8afb-444a-a236-8eed07c221bf service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:19:43.705214 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c14446-b4e4-47e5-ad41-b01e86931143 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:19:43.705868 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c14446-b4e4-47e5-ad41-b01e86931143 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:19:43.706773 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c14446-b4e4-47e5-ad41-b01e86931143 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:19:43.708021 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c14446-b4e4-47e5-ad41-b01e86931143 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:19:43.709867 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-57c14446-b4e4-47e5-ad41-b01e86931143 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:19:43.727486 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c14446-b4e4-47e5-ad41-b01e86931143 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:19:43.729299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c14446-b4e4-47e5-ad41-b01e86931143 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:19:43.745268 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1474/2945] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:19:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:19:48.573672 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c15e84-09ce-4a07-a000-94db54fadc7d service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:19:48.584078 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c15e84-09ce-4a07-a000-94db54fadc7d service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:19:48.587580 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7f315-d2d5-41fc-955e-26414fb1c0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:19:48.588217 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7f315-d2d5-41fc-955e-26414fb1c0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:19:48.588533 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7f315-d2d5-41fc-955e-26414fb1c0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:19:48.588813 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7f315-d2d5-41fc-955e-26414fb1c0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:19:48.590528 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d0d7f315-d2d5-41fc-955e-26414fb1c0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:19:48.604040 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7f315-d2d5-41fc-955e-26414fb1c0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:19:48.605490 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7f315-d2d5-41fc-955e-26414fb1c0f8 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:19:48.619012 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1472/2946] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:19:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:19:49.788166 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c14446-b4e4-47e5-ad41-b01e86931143 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:19:49.798142 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c14446-b4e4-47e5-ad41-b01e86931143 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:19:49.802594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f76e61-d00e-46ca-9464-2b88429aa640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:19:49.803235 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f76e61-d00e-46ca-9464-2b88429aa640 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:19:49.803523 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f76e61-d00e-46ca-9464-2b88429aa640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:19:49.803805 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f76e61-d00e-46ca-9464-2b88429aa640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:19:49.805594 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a1f76e61-d00e-46ca-9464-2b88429aa640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:19:49.819384 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f76e61-d00e-46ca-9464-2b88429aa640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:19:49.820881 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f76e61-d00e-46ca-9464-2b88429aa640 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:19:49.832416 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1475/2947] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:19:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:19:58.611421 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=18 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:19:59.827037 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=29 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:20:48.037434 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7f315-d2d5-41fc-955e-26414fb1c0f8 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:20:48.048367 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7f315-d2d5-41fc-955e-26414fb1c0f8 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:20:48.052340 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fccbad-c53f-4719-a86d-2d06b9f76c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0edda384caf46819169b69b858afc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:20:48.053046 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fccbad-c53f-4719-a86d-2d06b9f76c62 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:20:48.053326 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fccbad-c53f-4719-a86d-2d06b9f76c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:20:48.053598 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fccbad-c53f-4719-a86d-2d06b9f76c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:20:48.055860 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b2fccbad-c53f-4719-a86d-2d06b9f76c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:20:48.065789 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fccbad-c53f-4719-a86d-2d06b9f76c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:20:48.067917 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fccbad-c53f-4719-a86d-2d06b9f76c62 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:20:48.081315 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1473/2948] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:20:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 18:20:58.077176 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:21:00.469414 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f76e61-d00e-46ca-9464-2b88429aa640 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:21:00.486169 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f76e61-d00e-46ca-9464-2b88429aa640 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:21:00.491428 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3d491-e229-4b2d-b7dc-619ac29737d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:21:00.492086 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3d491-e229-4b2d-b7dc-619ac29737d1 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:21:00.492398 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3d491-e229-4b2d-b7dc-619ac29737d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:21:00.492673 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3d491-e229-4b2d-b7dc-619ac29737d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:21:00.495093 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fbb3d491-e229-4b2d-b7dc-619ac29737d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:21:00.508006 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3d491-e229-4b2d-b7dc-619ac29737d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:21:00.509707 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3d491-e229-4b2d-b7dc-619ac29737d1 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:21:00.521261 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1476/2949] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:21:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:21:02.776978 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fccbad-c53f-4719-a86d-2d06b9f76c62 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:21:02.791038 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fccbad-c53f-4719-a86d-2d06b9f76c62 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:21:02.797379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2be66-8367-4ad0-93b2-0199333adf16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380d6fe7057d44bfb20b0ae2bb1e664f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:21:02.798258 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2be66-8367-4ad0-93b2-0199333adf16 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:21:02.798597 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2be66-8367-4ad0-93b2-0199333adf16 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:21:02.799019 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2be66-8367-4ad0-93b2-0199333adf16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:21:02.801425 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d9a2be66-8367-4ad0-93b2-0199333adf16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:21:02.814716 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2be66-8367-4ad0-93b2-0199333adf16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:21:02.816284 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2be66-8367-4ad0-93b2-0199333adf16 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:21:02.828379 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1474/2950] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:21:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:21:07.118600 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3d491-e229-4b2d-b7dc-619ac29737d1 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:21:07.121165 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fbb3d491-e229-4b2d-b7dc-619ac29737d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:21:07.136729 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3d491-e229-4b2d-b7dc-619ac29737d1 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:21:07.142708 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb5a2e-af50-462c-8aea-b038948e4d9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:21:07.143363 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb5a2e-af50-462c-8aea-b038948e4d9e service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:21:07.143643 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb5a2e-af50-462c-8aea-b038948e4d9e service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:21:07.143916 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb5a2e-af50-462c-8aea-b038948e4d9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:21:07.145859 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0efb5a2e-af50-462c-8aea-b038948e4d9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:21:07.158835 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb5a2e-af50-462c-8aea-b038948e4d9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:21:07.161299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb5a2e-af50-462c-8aea-b038948e4d9e service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:21:07.190706 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1477/2951] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:21:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:21:10.599648 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2be66-8367-4ad0-93b2-0199333adf16 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:21:10.680065 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2be66-8367-4ad0-93b2-0199333adf16 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:21:10.685147 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:21:10.686014 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:21:10.686324 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:21:10.686600 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] PATH_INFO: `/v3/users/35918feccb574b978f06f71dc01912f2` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:21:10.687785 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35918feccb574b978f06f71dc01912f2)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:21:10.689729 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:21:10.735178 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35918feccb574b978f06f71dc01912f2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:10.735819 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35918feccb574b978f06f71dc01912f2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:10.736472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35918feccb574b978f06f71dc01912f2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:10.743298 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35918feccb574b978f06f71dc01912f2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:10.755695 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35918feccb574b978f06f71dc01912f2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:10.768738 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1475/2952] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:21:10 2022] DELETE /identity/v3/users/35918feccb574b978f06f71dc01912f2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:21:10.940699 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb5a2e-af50-462c-8aea-b038948e4d9e service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:21:10.952873 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb5a2e-af50-462c-8aea-b038948e4d9e service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:21:11.010009 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:21:11.011032 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:21:11.011320 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:21:11.011601 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] PATH_INFO: `/v3/projects/d026c3b708b4445da50aa0be2c02a8fa` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:21:11.012800 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d026c3b708b4445da50aa0be2c02a8fa)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:21:11.014695 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:21:11.061244 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd026c3b708b4445da50aa0be2c02a8fa', 'request_id': 'req-932fc9a5-a725-4d94-8707-9475ef2f1939'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.067321 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd026c3b708b4445da50aa0be2c02a8fa', 'request_id': 'req-932fc9a5-a725-4d94-8707-9475ef2f1939'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.074299 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd026c3b708b4445da50aa0be2c02a8fa', 'request_id': 'req-932fc9a5-a725-4d94-8707-9475ef2f1939'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.097209 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] The token cache is being invalidate because project d026c3b708b4445da50aa0be2c02a8fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:21:11.097502 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.098128 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.099711 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1478/2953] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:21:10 2022] DELETE /identity/v3/projects/d026c3b708b4445da50aa0be2c02a8fa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:21:11.104304 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:21:11.109161 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:21:11.123000 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291766c2-319e-47c4-90f7-cd852eb1988d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:21:11.171047 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:21:11.184522 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:21:11.185392 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:21:11.185673 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:21:11.185947 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] PATH_INFO: `/v3/users/9b67df432e64439e8b81b105b4eb30c3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:21:11.187141 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b67df432e64439e8b81b105b4eb30c3)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:21:11.189089 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:21:11.235889 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b67df432e64439e8b81b105b4eb30c3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.236559 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b67df432e64439e8b81b105b4eb30c3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.237157 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b67df432e64439e8b81b105b4eb30c3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.243814 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b67df432e64439e8b81b105b4eb30c3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.255416 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b67df432e64439e8b81b105b4eb30c3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.268599 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1476/2954] 10.222.0.57 () {58 vars in 1223 bytes} [Thu Jul 21 18:21:11 2022] DELETE /identity/v3/users/9b67df432e64439e8b81b105b4eb30c3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:21:11.441604 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:21:11.443116 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:21:11.457027 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fc9a5-a725-4d94-8707-9475ef2f1939 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:21:11.504471 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:21:11.515619 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57de169446f4409f8d2242abb44a08b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee47bf64e42844579243da3bda418cc7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:21:11.516501 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:21:11.516782 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:21:11.517055 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] PATH_INFO: `/v3/projects/a4badcf42f6c4ac2ad4ab1f22617faf7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:21:11.518218 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4badcf42f6c4ac2ad4ab1f22617faf7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:21:11.520115 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:21:11.568174 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4badcf42f6c4ac2ad4ab1f22617faf7', 'request_id': 'req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.575091 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4badcf42f6c4ac2ad4ab1f22617faf7', 'request_id': 'req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.583463 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4badcf42f6c4ac2ad4ab1f22617faf7', 'request_id': 'req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.594977 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] The token cache is being invalidate because project a4badcf42f6c4ac2ad4ab1f22617faf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 18:21:11.595304 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.595951 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b5137ee0-0e8a-4e38-aa1f-8df6b841d621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 18:21:11.597531 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1479/2955] 10.222.0.57 () {58 vars in 1232 bytes} [Thu Jul 21 18:21:11 2022] DELETE /identity/v3/projects/a4badcf42f6c4ac2ad4ab1f22617faf7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 18:21:21.265031 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:21:21.592349 dv-cinder-825492-27-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 18:23:46.660025 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 18:23:46.661472 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:23:46.775370 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d22503-ccf3-42bd-8abd-b41382b68a6a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 18:23:46.834029 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba458108-136d-4da1-9a8f-a54f67be4eff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '901cf70064c34b63a0eda979a5ce77e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66d87a2609e44063a774cbfc545412dc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 18:23:46.834740 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458108-136d-4da1-9a8f-a54f67be4eff service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 18:23:46.835028 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458108-136d-4da1-9a8f-a54f67be4eff service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 18:23:46.835302 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458108-136d-4da1-9a8f-a54f67be4eff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 18:23:46.837184 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ba458108-136d-4da1-9a8f-a54f67be4eff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 18:23:46.849001 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba458108-136d-4da1-9a8f-a54f67be4eff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 18:23:46.850573 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba458108-136d-4da1-9a8f-a54f67be4eff service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 18:23:46.860265 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1477/2956] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Jul 21 18:23:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 18:23:56.856123 dv-cinder-825492-27-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=76 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}